Benchmarks

Generated by npm run bench:page from the run of 2026-09-09 at commit 5bc506c. Do not edit by hand.

B1 has not run. The agent-cost axis reports not-run: not selected by --axis. Its two rows below read unmeasured, and they will keep reading unmeasured until it runs. They are not estimates, and there are no estimates on this page.

Published claims, settled

Every number this repository states in public, checked against the record that settles it. "Not met" and "unmeasured" are different outcomes and never collapse into each other: one says we measured and it is not true, the other says we have not measured.

B2 — cold start

Spawned processes, not an in-process micro-benchmark: process time is what a user and an agent actually pay, and excluding Node's own startup would flatter every row. bare node is the floor row and most of every other number.

These milliseconds are a property of the machine that ran them — AMD EPYC 7763 64-Core Processor, 4 cores, linux/x64, Node v24.13.0, on CI — and are not comparable across machines. Nothing gates on them. What is gated and banded is the ratio below, taken between two spawns interleaved in the same run, which cancels the machine out.

B1, the half that needs no model — what an agent can act on

Not B1, and not a stand-in for it: this measures nothing about tokens or turns. It measures whether the CLI's answer is legible to an agent, which needs no model at all. Ten tasks per variant, one spawn each, non-TTY with stdin closed — the only environment an agent gets. The same demo program, built on each engine.

Exit code is the one that decides an agent's next move: 2 means rewrite the command, any other non-zero means the command was fine and the world was not. Both incumbents answer 1 to a usage error, which tells an agent nothing — so it retries a malformed command until it gives up.

Bytes is reported against us and is not gated. commander reads fewer than we do, because our errors carry a hint naming the fix. That is a trade — bytes per failure against failed turns — and only B1 proper can settle it. It is on this page precisely so the trade is visible rather than quietly omitted.

B3 — compatibility

Read from compat-oracle and never recomputed here: two implementations of a compatibility rate produce two rates. The full method, and the control column that proves the gate works, are on Compatibility.

B4 — weight

Bundled is what your application grows by when you import that entry point: a fixture importing exactly it, one symbol used, esbuild-minified — the incumbent measured by the same command over the same fixture shape. A tarball size answers a different question.

Installed is what npm install puts on disk, including everything the package drags in. Ours drag nothing, which is where the two columns diverge most; but one of our packages serves every entry point it publishes, so the installed column is not per-entry-point and nothing gates on it.

Reproduce

npm install
npx turbo run build
npm run bench -- --check

Results are written to benchmarks/results/<suite>/<date>.json and read by the Stage 6 control bands in .sdlc/bands/control-bands.json. The suite refuses to emit a band value it did not measure; see benchmarks/README.md.