Validators

Validators are the quality gates in dmx. They run automatically at loop boundaries and determine whether a loop can complete.

What validators are

A validator is a subprocess that receives the loop context as JSON on stdin and returns a structured result as JSON on stdout. It exits 0 on pass, 1 on failure.

Validators are configured in loop YAML under validators:. Each entry specifies:

• tool: the validator to run, resolved to validators/{tool}.py in the repo, or the bundled fallback
• checks: named checks within the validator, each with a required flag

Required checks must pass for the loop to complete. Optional checks produce a warning on failure.

Bundled validators

dmx ships five validators covering the common quality gates.

check_spec_complete

Used by: Spec Loop

Checks that spec.md is complete:

• spec_exists: file exists and is non-empty
• qa_answered: Q&A answers are filled in, not placeholder text
• technical_approach_filled: Technical Approach section has meaningful content
• scope_defined: Scope section exists and is not empty

check_plan_complete

Used by: Plan Loop

Checks that tasks.md is a valid plan:

• tasks_file_exists: file exists and is non-empty
• phases_defined: at least one phase is defined
• tasks_have_descriptions: tasks have content, not empty checkboxes

run_tests

Used by: Dev Loop, Validate Loop

Detects and runs your project's test suite:

• tests_pass: the test command exits 0
• coverage_threshold: optional; not measured by the bundled validator (override to add)

Auto-detects the test command:

• Python + pyproject.toml + uv.lock → uv run pytest -q
• Python + pyproject.toml (no uv.lock) → pytest -q
• Node.js + package.json test script → npm test --silent
• Makefile with a test: target → make test

spec_adherence

Used by: Dev Loop, Validate Loop

Grades a structured validation-report.json artifact, not free-text skill output. The validate skill diffs the branch against the spec, records its scope/regression/edge-case findings to .dmx/jobs/{job_id}/validation-report.json, and this validator reads that file:

• scope_matches_spec: fails if any scope item's verdict is missing, or scope_creep is non-empty. partial verdicts pass but are surfaced in the message.
• edge_cases_addressed: fails only if an edge case is explicitly flagged addressed: false.
• no_regressions: fails only if the report's regressions list is non-empty.

If the report is missing, malformed, or stale (its recorded commit doesn't match current HEAD), every check fails with a message to re-run /dmx/validate. Grading a structured diff report, rather than the agent's free-text summary, keeps the check tied to what's actually in the diff. See Writing Validators.

check_pr_ready

Used by: Release Loop

Checks PR and ticket state:

• pr_exists: a PR has been opened for the current branch (via gh pr view)
• ticket_transitioned: the bundled default cannot verify real ticket status without ticketing API credentials, so it passes unconditionally with a note to confirm manually (or when no ticketing is configured). Override this validator for a real, API-backed check.
• memory_updated: optional; passes if the latest commit touched a .dmx/*.md file, or if activeContext.md exists at all

Required vs. optional checks

The required flag on each check determines what happens on failure:

required	Failure behavior
true	Loop pauses or fails (per failure_handling). Job cannot advance until resolved.
false	Warning is emitted. Job advances per on_optional_failure policy.

Failure handling

When required checks fail, the loop applies failure_handling:

Policy	What happens
pause	Loop pauses. You fix the issue and call /dmx/loop-continue to retry validators.
fail	Loop terminates. The job is marked failed.

The default for all five bundled loops is pause.

Resolution order

dmx looks for validators in this order:

1. App repo: validators/{tool}.py at the repository root
2. Bundled: the validator shipped with dmx

A file at validators/run_tests.py in your repo overrides the bundled run_tests validator. You own the implementation; dmx owns the contract.

Writing custom validators

See Writing Validators for the full contract and examples.