Validators

Validators are the quality gates in dmx. They run automatically at loop boundaries and determine whether a loop can complete.

What validators areCopy Anchor Link

A validator is a subprocess that receives the loop context as JSON on stdin and returns a structured result as JSON on stdout. It exits 0 on pass, 1 on failure.

Validators are configured in loop YAML under validators:. Each entry specifies:

• tool: the validator to run, resolved to validators/{tool}.py in the repo, then any shared sources, then the bundled fallback
• checks: named checks within the validator, each with a required flag
• timeout_seconds: optional; how long the validator may run. Defaults to 630 seconds, minimum 60. See timeout_seconds.

Required checks must pass for the loop to complete. Optional checks produce a warning on failure.

Validators run in the background, so a long test suite doesn't time out your IDE's connection to dmx. While they run, the loop's status is validating and the agent waits for the result with the loop_status tool.

Bundled validatorsCopy Anchor Link

dmx ships five validators covering the common quality gates.

check_spec_completeCopy Anchor Link

Used by: Spec Loop

Checks that spec.md is complete:

• spec_exists: file exists and is non-empty
• qa_answered: Q&A answers are filled in, not placeholder text
• technical_approach_filled: Technical Approach section has meaningful content
• scope_defined: Scope section exists and is not empty
• spec_identity_matches_branch: the branch in the spec's frontmatter is the branch you're on. This stops a spec left over from another ticket from being filed under the wrong job.

check_plan_completeCopy Anchor Link

Used by: Plan Loop

Checks that tasks.md is a valid plan:

• tasks_file_exists: file exists and is non-empty
• phases_defined: at least one phase is defined
• tasks_have_descriptions: tasks have content, not empty checkboxes

run_testsCopy Anchor Link

Used by: Dev Loop, Validate Loop

Detects and runs your project's test suite:

• tests_pass: the test command exits 0. On failure, the message includes the last 20 lines of the test output.

The bundled validator doesn't measure coverage. To enforce a threshold, override run_tests and add a coverage check to your own loop YAML.

The test command gets 30 seconds less than the validator's timeout_seconds, so a slow suite is reported as a test timeout rather than a validator timeout. The bundled Dev and Validate Loops set timeout_seconds: 630, which gives the suite 600 seconds.

Auto-detects the test command:

• Python + pyproject.toml + uv.lock → uv run pytest -q
• Python + pyproject.toml (no uv.lock) → pytest -q
• Node.js + package.json test script → npm test --silent
• Makefile with a test: target → make test

spec_adherenceCopy Anchor Link

Used by: Validate Loop

Grades a structured validation-report.json artifact, not free-text skill output. The validate skill diffs the branch against the spec, records its scope/regression/edge-case findings to .dmx/jobs/{job_id}/validation-report.json, and this validator reads that file:

• scope_matches_spec: fails if any scope item's verdict is missing, or scope_creep is non-empty. partial verdicts pass but are surfaced in the message.
• edge_cases_addressed: fails only if an edge case is explicitly flagged addressed: false.
• no_regressions: fails only if the report's regressions list is non-empty.

If the report is missing, malformed, or stale (its recorded commit doesn't match current HEAD), every check fails with a message to re-run /dmx/validate. Grading a structured diff report, rather than the agent's free-text summary, keeps the check tied to what's actually in the diff. See Writing Validators.

check_pr_readyCopy Anchor Link

Used by: Release Loop

Checks PR and ticket state:

• pr_exists: a PR has been opened for the current branch (via gh pr view)
• ticket_transitioned: the bundled default cannot verify real ticket status without ticketing API credentials, so it passes unconditionally with a note to confirm manually (or when no ticketing is configured). Override this validator for a real, API-backed check.
• memory_updated: optional; fails if anything under .dmx/ has uncommitted changes, staged or not, so memory bank edits a skill forgot to commit can't silently drop out of the PR. .dmx/jobs/ is excluded — that's the loop runtime's own state, and it's routinely uncommitted while a loop is paused. Once the tree is clean, it passes if the latest commit touched a .dmx/*.md file, or if activeContext.md exists

Required vs. optional checksCopy Anchor Link

The required flag on each check determines what happens on failure:

Failure handlingCopy Anchor Link

When required checks fail, the loop applies failure_handling:

The default for all five bundled loops is pause.

When a loop pauses on a failed check, or finishes with a warning, its reply says why. It shows each validator's message and names every failed check with that check's own message, such as the end of a test run or a timeout reason. A required check the validator didn't report at all counts as failed and is named too. loop_status returns the same explanation.

TimeoutsCopy Anchor Link

Each validator has a time limit: its timeout_seconds, or 630 seconds if the loop doesn't set one. When a validator runs past it, dmx stops the validator and every process it started (SIGTERM, then SIGKILL after 5 seconds; on Windows, taskkill /T /F), and the check fails with a message naming the limit and where to change it. A process that starts its own session, such as some Docker-based test setups, isn't stopped.

Resolution orderCopy Anchor Link

dmx looks for validators in this order:

1. App repo: validators/{tool}.py at the repository root
2. Shared sources: .dmx/vendor/{name}/validators/{tool}.py for each source declared in .dmx/shared-sources.yaml, in declared order
3. Bundled: the validator shipped with dmx

A file at validators/run_tests.py in your repo overrides the bundled run_tests validator, and any shared-source version of it. You own the implementation; dmx owns the contract.

Shared sources let an organization maintain validators once and vendor them into every repo. See Shared Sources.

Writing custom validatorsCopy Anchor Link

See Writing Validators for the full contract and examples.