# Benchmarks

> Every number this project claims in public, measured — and the ones that are not measured, saying so.

Source: https://burgee.interlace.tools/docs/benchmarks

Generated by `npm run bench:page` from the run of 2026-09-09 at commit `5bc506c`.
Do not edit by hand.

> **B1 has not run.** The agent-cost axis reports **not-run**: not selected by --axis.
> Its two rows below read *unmeasured*, and they will keep reading *unmeasured* until it runs.
> They are not estimates, and there are no estimates on this page.

## Published claims, settled

Every number this repository states in public, checked against the record that settles it.
**"Not met" and "unmeasured" are different outcomes** and never collapse into each other:
one says we measured and it is not true, the other says we have not measured.

| Claim | Target | Measured | |
| :--- | :--- | ---: | :--- |
| the engine starts at or below cac, the lightest framework in the landscape | `<= 1` | 1.322 ratio | ❌ **not met** |
| the core entry point is under 52 KB bundled | `<= 53248` | 34,841 bytes | ✅ met |
| `burgee` is lighter in a user's bundle than `cac`, the package it replaces | `<= 1` | 3.333 ratio | ❌ **not met** |
| `burgee/commander` is lighter in a user's bundle than `commander`, the package it replaces | `<= 1` | 1.496 ratio | ❌ **not met** |
| `burgee/yargs` is lighter in a user's bundle than `yargs`, the package it replaces | `<= 1` | 0.906 ratio | ✅ met |
| `roundel/chalk` is lighter in a user's bundle than `chalk`, the package it replaces | `<= 1` | 0.575 ratio | ✅ met |
| `flagstaff/ora` is lighter in a user's bundle than `ora`, the package it replaces | `<= 1` | 0.797 ratio | ✅ met |
| `flagstaff/boxen` is lighter in a user's bundle than `boxen`, the package it replaces | `<= 1` | 0.301 ratio | ✅ met |
| `flagstaff/log-update` is lighter in a user's bundle than `log-update`, the package it replaces | `<= 1` | 0.315 ratio | ✅ met |
| commander's own test suite passes 1360 of 1360 against our entry point | `>= 1360` | 1,360 tests | ✅ met |
| yargs's own test suite passes 804 of 804 against our entry point | `>= 804` | 804 tests | ✅ met |
| chalk's own test suite passes 58 of 58 against our entry point | `>= 58` | 58 tests | ✅ met |
| ora's own test suite passes 99 of 99 against our entry point | `>= 99` | 99 tests | ✅ met |
| log-update's own test suite passes 99 of 99 against our entry point | `>= 99` | 99 tests | ✅ met |
| boxen's own test suite passes 84 of 84 against our entry point | `>= 84` | 84 tests | ✅ met |
| an agent spends at least 40% fewer tokens per task against a CLI that meets the floor | `<= 0.6` | — | **unmeasured** — not selected by --axis |
| an agent takes at least 30% fewer turns per task against a CLI that meets the floor | `<= 0.7` | — | **unmeasured** — not selected by --axis |

## B2 — cold start

Spawned processes, not an in-process micro-benchmark: process time is what a user and an
agent actually pay, and excluding Node's own startup would flatter every row.
**`bare node` is the floor row** and most of every other number.

> These milliseconds are a property of the machine that ran them —
> AMD EPYC 7763 64-Core Processor, 4 cores, linux/x64, Node v24.13.0, on CI — and are **not comparable across
> machines**. Nothing gates on them. What is gated and banded is the ratio below, taken
> between two spawns interleaved in the same run, which cancels the machine out.

| Variant | p50 ms | p95 ms | spawns |
| :--- | ---: | ---: | ---: |
| `bare node` | 26.9 | 27.7 | 42 |
| `commander` | 42.1 | 46.7 | 42 |
| `burgee/commander` | 46.5 | 49.8 | 42 |
| `yargs` | 105.4 | 110.6 | 42 |
| `burgee/yargs` | 86.5 | 90.3 | 42 |
| `cac` | 30.9 | 32.4 | 42 |
| `burgee` | 40.9 | 44.2 | 42 |

| Ratio | p50 | p95 | gate |
| :--- | ---: | ---: | ---: |
| burgee/commander ÷ commander | **1.078×** | 1.181× | ≤ 1.4 |
| burgee/yargs ÷ yargs | **0.822×** | 0.862× | ≤ 0.95 |
| burgee ÷ cac | **1.322×** | 1.422× | ≤ 1.7 |

## B1, the half that needs no model — what an agent can act on

Not B1, and not a stand-in for it: this measures nothing about tokens or turns. It measures
whether the CLI's answer is *legible* to an agent, which needs no model at all. Ten tasks
per variant, one spawn each, non-TTY with **stdin closed** — the only environment an agent
gets. The same demo program, built on each engine.

| Variant | hangs/100 | exit code | `--json` | bytes |
| :--- | ---: | ---: | ---: | ---: |
| `burgee` | 0 | 100.0% | 100.0% | 83 |
| `commander` | 0 | 40.0% | 25.0% | 31 |
| `yargs` | 0 | 40.0% | 25.0% | 458 |

**Exit code** is the one that decides an agent's next move: `2` means *rewrite the command*,
any other non-zero means *the command was fine and the world was not*. Both incumbents
answer `1` to a usage error, which tells an agent nothing — so it retries a malformed
command until it gives up.

**Bytes is reported against us and is not gated.** commander reads fewer than we do, because
our errors carry a `hint` naming the fix. That is a trade — bytes per failure against failed
turns — and only B1 proper can settle it. It is on this page precisely so the trade is
visible rather than quietly omitted.

## B3 — compatibility

Read from `compat-oracle` and never recomputed here: two implementations of a
compatibility rate produce two rates. The full method, and the control column that proves
the gate works, are on [Compatibility](/docs/compatibility).

| Host | Passing | Rate |
| :--- | ---: | ---: |
| **commander** | 1,360 | 100.0% |
| **yargs** | 804 | 100.0% |
| **chalk** | 58 | 100.0% |
| **ora** | 99 | 100.0% |
| **log-update** | 99 | 100.0% |
| **boxen** | 84 | 100.0% |

## B4 — weight

**Bundled** is what your application grows by when you import that entry point: a fixture
importing exactly it, one symbol used, esbuild-minified — the incumbent measured by the
same command over the same fixture shape. A tarball size answers a different question.

**Installed** is what `npm install` puts on disk, including everything the package drags
in. Ours drag nothing, which is where the two columns diverge most; but one of our packages
serves every entry point it publishes, so the installed column is not per-entry-point and
nothing gates on it.

| Entry point | Replaces | Bundled | Theirs | Ratio | Installed | Theirs |
| :--- | :--- | ---: | ---: | ---: | ---: | ---: |
| `burgee` | `cac` | 34,841 | 10,452 | **3.333×** | 574,318 | 41,198 |
| `burgee/commander` | `commander` | 58,458 | 39,084 | **1.496×** | 574,318 | 207,368 |
| `burgee/yargs` | `yargs` | 100,731 | 111,152 | **0.906×** | 574,318 | 526,898 |
| `roundel/chalk` | `chalk` | 4,594 | 7,990 | **0.575×** | 47,122 | 56,029 |
| `flagstaff/ora` | `ora` | 47,424 | 59,519 | **0.797×** | 244,079 | 284,554 |
| `flagstaff/boxen` | `boxen` | 19,531 | 64,823 | **0.301×** | 244,079 | 747,463 |
| `flagstaff/log-update` | `log-update` | 11,460 | 36,390 | **0.315×** | 244,079 | 261,977 |

## Reproduce

```bash
npm install
npx turbo run build
npm run bench -- --check
```

Results are written to `benchmarks/results/<suite>/<date>.json` and read by the Stage 6
control bands in `.sdlc/bands/control-bands.json`. The suite refuses to emit a band value
it did not measure; see `benchmarks/README.md`.