wednesday docs
Docs/Core concepts

Verification

Verification is authoritative. The tests decide when the work is done, not the model. This page is the policy in detail.

The tests are the authority#

Wednesday runs the verification commands declared with the task or by the crew. A failing command blocks the result, triggers repair, and ends the run as failed if the budget runs out. A model cannot talk its way past a red test.

This is the core difference between a model that writes code and a team that ships checked code.

What counts as a gate#

Verification is not only npm test. If the workspace declares a gate of its own, Wednesday runs it too:

a run with two gates
Verification:
  $ npm test
  $ npm run typecheck        <- picked up from package.json

Recognised shapes include npm test, npm run typecheck, tsc, and lint. Only scripts the project already declares are used — nothing is guessed and nothing is downloaded. A small model’s most common defect is a syntax slip, and the compiler is the cheapest deterministic reader of one.

When no verification plan exists#

A task may declare no test commands. The tester then passes with a note — No verification plan provided — skipping verification — and the run continues. It is not failed for the absence of a plan.

The plan is still authoritative when it exists: a declared command that fails still blocks, still triggers repair, and still ends the task as failed.

The repair loop#

Failures go back to the crew with the failure output. The loop is capped by maxRepairAttempts and constrained by the token and dollar budgets. It stops when the gate passes or the budget is exhausted; it never loops forever.

crew policy
wednesday team set repairs 2
wednesday team set tokens 200000
wednesday team set usd 1.50

The security scan#

Every run also passes a security scan. It matches shapes — a leaked secret, a dangerous call — and a pass means “no secret shape, no dangerous call shape”, never “no vulnerability”.

The same rules ship as a command you can run yourself: wednesday scan. wednesday scan --install-hook writes a pre-commit hook that scans what is staged, so the gate travels with the repository. See Security & data.

The receipt#

Tokens, wall-clock time and dollars are recorded per seat. With one model wearing several name tags, a per-model total could not tell you that the same model did three jobs. For a locally hosted model the real cost is GPU-minutes, so wall clock is on the same line.

Wednesday will not invent a price. It knows a free tier costs nothing and it knows nothing else, because your contract is what you pay. Set your own price and the receipts become real:

pricing
# USD per million tokens: input,output
FACTORY_PRICE_QWEN_QWEN3_CODER=0.30,1.20
FACTORY_PRICE_OPENROUTER_FREE=0,0

The model name is uppercased and every non-alphanumeric character becomes _. A model with no price is reported as unpriced: its tokens still count and the dollar column stays empty rather than showing a made-up figure. A local model shows $0.0000.