Build and run
A pipeline is built before it is run, and the two are genuinely different operations.
- Build proves every component's body on a sample of the data, end to end, and pins the result. For a node that authors its own code, building is where that code is written and proven.
- Run executes those pinned bodies over the full corpus. No authoring happens at run time.
Building is where iteration is cheap. Running is where the work is real. Keeping them apart is what makes it affordable to get a pipeline wrong several times before getting it right.
Verification by construction
Verification by construction beats verification by inspection.
Checks fire at each junction while a pipeline is being assembled — a component's contract is proven when it is authored, the graph is validated before it is built, and each component is accepted before the next is written. Invalid work is pruned as it appears, so a pipeline that finishes building could not have been assembled any other way. A check applied only at the end can report that something is wrong once the work is already done — strictly less useful and strictly more expensive.
Lifecycle
edit / the Architect proposes
(nothing) ──────────────────────────────────▶ DRAFT
│ Build
▼
BUILDING ────── Stop ──▶ cancelled
│ every node authored, proven, pinned
▼
VERIFYING ◀── the Architect reads the sample output
│ │
not sensible ──┘ └── sensible
(re-author culprit) │
▲ ▼
└────────────────── BUILT
│ Run
▼
RUNNING ──── Stop ──▶ cancelled
│ full corpus
▼
COMPLETE
Editing a node, or rebuilding one, marks the pipeline dirty — the build no longer matches the graph, and running is disabled until it is built again. The gating is strict on purpose: you cannot run something unbuilt, and you cannot run something stale.
Verification closes the loop
A successful build proves each component runs. It does not prove the pipeline produces something sensible. So when a build finishes, the pipeline's terminal output from the sample pass is judged: does it parse, does it have the expected shape, is it non-empty, is it plausible? If it is sensible, the pipeline becomes runnable. If not, the component at fault is sent back for revision, seeded with the bad output as context, and the loop repeats — bounded — until it verifies.
This is the cheap half of the work: iterate on a little data until the result makes sense, then run the full corpus once.
When a run fails
A failure during a real run is triaged by scope rather than reflexively rewriting code:
| Verdict | Meaning |
|---|---|
| re-author | the logic is wrong — revise this component's body |
| amend config | the logic is fine, a value is wrong |
| rewire | the graph is wrong — a structural change, producing a new version |
| report infra | not a pipeline problem at all |
Only the reversible verdict applies itself; the expensive ones are recommendations. An unclassifiable failure degrades to report infra — a transient outage is not a code defect, and re-authoring identical code against a broken dependency will fail identically.
Revision starts from what exists
Pinning code is not final. A component can pass on a sample and still fail at corpus scale, or underperform, or simply become the wrong thing when a requirement changes. Any component can return to authoring, and when it does, it carries its previous body, its brief, and the trigger — the error, the disappointing metric, or the new requirement. Revision does not start from a blank page.