Compatibility
Each host's own test suite, run against burgee in CI. A published pass rate that only goes up — never the word "compatible".
Generated by npm run compat:page from the oracle's last run on 2026-10-19.
Do not edit by hand.
Each row is the host's own test suite — vendored, unmodified apart from the import specifier — pointed at burgee's front-end through a generated shim. The control column is the same suite pointed at the real host: it proves the gate works before it grades anything of ours, and it is the reference total every rate is measured against. A file that fails to import registers as one test instead of its twenty, so measuring against registered tests would flatter a partial implementation; measuring against the control's total does not.
| Host | Front-end | burgee | rate | control | rate | internals (burgee) | internals (control) |
|---|---|---|---|---|---|---|---|
| commander | burgee/commander | 1360 / 1360 | 100.0% | 1360 / 1360 | 100.0% | 12 / 12 | 12 / 12 |
| extra-typings | burgee/commander/extra-typings | 369 / 369 | 100.0% | 369 / 369 | 100.0% | — | — |
| yargs | burgee/yargs | 816 / 816 | 100.0% | 814 / 816 | 99.8% | 23 / 23 | 23 / 23 |
| chalk | roundel/chalk | 59 / 59 | 100.0% | 59 / 59 | 100.0% | — | — |
| picocolors | roundel | 1 / 1 | 100.0% | 1 / 1 | 100.0% | — | — |
| supports-color | roundel | 55 / 55 | 100.0% | 55 / 55 | 100.0% | — | — |
| yoctocolors | roundel | 78 / 78 | 100.0% | 78 / 78 | 100.0% | — | — |
| ora | flagstaff/ora | 99 / 99 | 100.0% | 99 / 99 | 100.0% | — | — |
| log-update | flagstaff/log-update | 99 / 99 | 100.0% | 99 / 99 | 100.0% | — | — |
| boxen | flagstaff/boxen | 213 / 213 | 100.0% | 213 / 213 | 100.0% | — | — |
| cli-table3 | flagstaff/cli-table3 | 29 / 29 | 100.0% | 29 / 29 | 100.0% | 103 / 104 | 103 / 104 |
| string-width | linegauge | 233 / 233 | 100.0% | 233 / 233 | 100.0% | — | — |
| strip-ansi | linegauge/strip | 8 / 8 | 100.0% | 8 / 8 | 100.0% | — | — |
| cross-spawn | bellpull/cross-spawn | 68 / 68 | 100.0% | 68 / 68 | 100.0% | — | — |
| which | bellpull | 5 / 5 | 100.0% | 5 / 5 | 100.0% | — | — |
| execa | bellpull | 0 / 1360 | 0.0% | 1360 / 1360 | 100.0% | — | — |
| rc | seniority/rc | 1 / 1 | 100.0% | 1 / 1 | 100.0% | — | — |
| wrap-ansi | linegauge/wrap | 85 / 85 | 100.0% | 85 / 85 | 100.0% | — | — |
| slice-ansi | linegauge/slice | 104 / 104 | 100.0% | 104 / 104 | 100.0% | — | — |
| cli-truncate | linegauge | 12 / 12 | 100.0% | 12 / 12 | 100.0% | — | — |
| widest-line | linegauge | 1 / 1 | 100.0% | 1 / 1 | 100.0% | — | — |
| cosmiconfig | seniority | 242 / 243 | 99.6% | 242 / 243 | 99.6% | 10 / 10 | 0 / 10 |
| lilconfig | seniority | 77 / 77 | 100.0% | 77 / 77 | 100.0% | — | — |
| dotenv | seniority | 193 / 193 | 100.0% | 193 / 193 | 100.0% | 4 / 10 | 4 / 10 |
| find-up | seniority | 104 / 104 | 100.0% | 104 / 104 | 100.0% | — | — |
| clack | caique/clack | 16 / 16 | 100.0% | 16 / 16 | 100.0% | — | — |
| inquirer-prompts | caique/inquirer | 213 / 213 | 100.0% | 213 / 213 | 100.0% | — | — |
| inquirer-core | caique/inquirer | 41 / 41 | 100.0% | 41 / 41 | 100.0% | — | — |
| meow | burgee/meow | 146 / 148 | 98.6% | 146 / 148 | 98.6% | — | — |
| ansi-escapes | paratext | 4 / 4 | 100.0% | 4 / 4 | 100.0% | — | — |
| terminal-link | paratext/terminal-link | 8 / 8 | 100.0% | 8 / 8 | 100.0% | — | — |
| term-img | paratext/term-img | 18 / 18 | 100.0% | 18 / 18 | 100.0% | — | — |
| restore-cursor | closeout | 6 / 6 | 100.0% | 6 / 6 | 100.0% | — | — |
| exit-hook | closeout | 21 / 21 | 100.0% | 21 / 21 | 100.0% | — | — |
| signal-exit | closeout | 134 / 135 | 99.3% | 134 / 135 | 99.3% | — | — |
| onetime | closeout | 6 / 6 | 100.0% | 6 / 6 | 100.0% | — | — |
| cli-cursor | closeout | 8 / 8 | 100.0% | 8 / 8 | 100.0% | — | — |
| ink | controlroom/ink | 1304 / 1304 | 100.0% | 1303 / 1304 | 99.9% | 137 / 148 | 332 / 332 |
| inkjs-ui | controlroom/ink | 102 / 102 (7 the host marks failing and we pass) | 100.0% | 102 / 102 | 100.0% | — | — |
Every file of every suite is vendored and run, and every case that leaves the gate is named below. Files that import only the host's internal modules (its own file layout) are graded on the informational internals columns and never enter the gate: passing them would mean copying the host, not being compatible with it. A public-surface file that also touches an internal module stays in the gate, with that import shimmed to our main entry.
Cases excluded from the gate
A case that cannot fail for any implementation measures nothing, and counting it inflates the rate. These are excluded by name — they still run, and they still appear in the raw TAP.
| Host | Cases | Why |
|---|---|---|
| cli-table3 | test/verify-legacy-compatibility-test.js > verify original cli-table behavior > | Nine cases that call commonTests(require('cli-table')) — the legacy incumbent, not the target. They pass whatever COMPAT_TARGET names: breaking the target completely still scored 10 / 33, because 27% of the row was a self-test of a third-party package. The other nine, @api cli-table2 matches verified behavior, run the same assertions through the shim and are gated. cli-table stays a devDependency so the file still loads and the excluded cases still run; they are subtracted from the number, not hidden from the TAP. |
| execa | write to fast-exit subprocess | return/early-error.js runs execa(`fast-exit-${process.platform}`, …) — a compiled executable upstream ships as test/fixtures/fast-exit-darwin (Mach-O) and fast-exit-linux (ELF). The vendor step never copies a compiled executable (binaries.ts; both are listed under omitted in vendor/execa/PROVENANCE), so the case cannot pass for anyone and is excluded for ours and the control alike. It was the one case the suite guards with the CPU (if (!isWindows), then if (arch() === 'x64')), so the reference is now 1179 on every machine rather than 1180 on ubuntu and 1179 on an arm64 Mac. bellpull failed it where it ran, so the target's number does not move. |
| lilconfig | > options > packageProp > string[] with null in the middle > async | Un-awaited expect(…).rejects.toThrowError: the assertion never settles before the case ends, so it passes vacuously under jest and errors under vitest 5. Its > sync sibling asserts the same thing synchronously and is gated. |
| lilconfig | > lilconfig > when to throw > non existing file | Un-awaited expect(…).rejects.toThrowError, twice — once against lilconfig, once against cosmiconfig. The lilconfigSync case of the same name is synchronous, real, and gated. |
| lilconfig | > lilconfig > when to throw > throws for invalid json | Un-awaited expect(…).rejects.toThrowError. The lilconfigSync case of the same name is gated. |
| lilconfig | > lilconfig > when to throw > throws for provided filepath that does not exist | Un-awaited expect(…).rejects.toThrowError. The lilconfigSync case of the same name is gated. |
| lilconfig | > lilconfig > when to throw > no loader specified for the search place | Un-awaited expect(…).rejects.toThrowError. The lilconfigSync case of the same name is gated. |
| lilconfig | > lilconfig > when to throw > loader is not a function | Un-awaited expect(…).rejects.toThrowError. The lilconfigSync case of the same name is gated. |
| lilconfig | > lilconfig > when to throw > throws for empty strings passed to load | Un-awaited expect(…).rejects.toThrowError. The lilconfigSync case of the same name is gated. |
| find-up | findup › async (dot file) | Asserts the nearest .git directory is the package's own repository root. A vendored copy has none, so the walk finds whatever repository holds the vendored copy: find-up 8.0.0 fails it here, the same way. |
| find-up | findup › sync (dot file) | The synchronous twin of async (dot file), with the same expectation of a .git at the vendored root, and the same failure against find-up 8.0.0 itself. |
| clack | test/path.test.ts > | 30 cases. Snapshots of the path prompt's frames, and the one file in this suite that fails against clack's own published package here: it mocks node:fs with vi.mock('node:fs') and no factory, which vitest answers from a __mocks__ directory beside the project root. Upstream's __mocks__/fs.cjs is vendored there by extraDirs and is still never loaded — measured 2026-09-14 by putting a console.error in it and watching it not print under vitest 5.0.0, which upstream's vitest 3.2.4 does load; a test.alias for node:fs is no better. So those 30 read the real filesystem, list the real /tmp, and diff against a memfs snapshot. That was a controlFailures allowance of 30 until 2026-09-20 and is now redundant: the file is subtracted as a drawing like the other sixteen, and an allowance that excuses a case nobody counts is a dial, which is the objection term-img's note already records. |
| clack | test/autocomplete.test.ts > | 27 cases, 24 snapshot assertions — the autocomplete prompt drawn frame by frame. |
| clack | test/box.test.ts > | 46 cases, 23 snapshot assertions — box borders, padding and title placement, drawn. |
| clack | test/confirm.test.ts > | 24 cases, 12 snapshot assertions — the confirm prompt drawn in each of its states. |
| clack | test/date.test.ts > | 16 cases, 7 snapshot assertions — the date prompt drawn. |
| clack | test/group-multi-select.test.ts > | 42 cases, 21 snapshot assertions — grouped selection drawn. |
| clack | test/log.test.ts > | 32 cases, 16 snapshot assertions — the log symbols and their bars, drawn. |
| clack | test/multi-line.test.ts > | 28 cases, 14 snapshot assertions — the multi-line editor drawn. |
| clack | test/multi-select.test.ts > | 48 cases, 24 snapshot assertions — multi-selection drawn. |
| clack | test/note.test.ts > | 18 cases, 9 snapshot assertions — the note box drawn. |
| clack | test/password.test.ts > | 18 cases, 9 snapshot assertions — the masked prompt drawn. |
| clack | test/progress-bar.test.ts > | 44 cases, 19 snapshot assertions — the progress bar drawn at each fill. |
| clack | test/select-key.test.ts > | 24 cases, 12 snapshot assertions — the key-select prompt drawn. |
| clack | test/select.test.ts > | 40 cases, 20 snapshot assertions — the select prompt drawn. |
| clack | test/spinner.test.ts > | 60 cases, 28 snapshot assertions — spinner frames, drawn. |
| clack | test/task-log.test.ts > | 64 cases, 27 snapshot assertions — the task log drawn as it fills and clears. |
| clack | test/text.test.ts > | 28 cases, 14 snapshot assertions — the text prompt drawn. |
| clack | test/guide.test.ts > guide > no prompt renders a guide when withGuide is globally false | One case, not a drawing, and it is excluded because it grades a package other than @clack/prompts. The test file itself runs import { updateSettings } from '@clack/core', calls it, and asserts that every prompt reads the module-level state of that sibling package. U6 forbids caique a runtime dependency, so no implementation of ours can see that state (D-152). The same setting through the public name is updateSettings, which @clack/prompts re-exports and caique/clack exports. packages/caique/src/clack.test.ts holds all twelve prompts to it. The sibling case no prompt renders a guide when withGuide is false stays graded and passes. A program that imports updateSettings from @clack/core directly still reaches clack's state, not caique's (D-20260930-caique-clack-core-exclusion). |
| inquirer-prompts | packages/editor/editor.test.ts > editor prompt > open editor after pressing enter | Asserts on, and answers through, vi.mock('@inquirer/external-editor')'s editAsync — a module of the incumbent's tree that caique/inquirer does not import (caique takes no dependency outside the burgee family, U6), so the mock can neither observe nor drive any implementation of ours. Excluded for both sides by exact title (D-20261011-inquirer-prompts-editor-mock). |
| inquirer-prompts | packages/editor/editor.test.ts > editor prompt > accepts explicit undefined as default | Asserts on, and answers through, vi.mock('@inquirer/external-editor')'s editAsync — a module of the incumbent's tree that caique/inquirer does not import (caique takes no dependency outside the burgee family, U6), so the mock can neither observe nor drive any implementation of ours. Excluded for both sides by exact title (D-20261011-inquirer-prompts-editor-mock). |
| inquirer-prompts | packages/editor/editor.test.ts > editor prompt > open editor immediately | Asserts on, and answers through, vi.mock('@inquirer/external-editor')'s editAsync — a module of the incumbent's tree that caique/inquirer does not import (caique takes no dependency outside the burgee family, U6), so the mock can neither observe nor drive any implementation of ours. Excluded for both sides by exact title (D-20261011-inquirer-prompts-editor-mock). |
| inquirer-prompts | packages/editor/editor.test.ts > editor prompt > allow setting a default value & postfix | Asserts on, and answers through, vi.mock('@inquirer/external-editor')'s editAsync — a module of the incumbent's tree that caique/inquirer does not import (caique takes no dependency outside the burgee family, U6), so the mock can neither observe nor drive any implementation of ours. Excluded for both sides by exact title (D-20261011-inquirer-prompts-editor-mock). |
| inquirer-prompts | packages/editor/editor.test.ts > editor prompt > allow setting temp file options | Asserts on, and answers through, vi.mock('@inquirer/external-editor')'s editAsync — a module of the incumbent's tree that caique/inquirer does not import (caique takes no dependency outside the burgee family, U6), so the mock can neither observe nor drive any implementation of ours. Excluded for both sides by exact title (D-20261011-inquirer-prompts-editor-mock). |
| inquirer-prompts | packages/editor/editor.test.ts > editor prompt > handles validation | Asserts on, and answers through, vi.mock('@inquirer/external-editor')'s editAsync — a module of the incumbent's tree that caique/inquirer does not import (caique takes no dependency outside the burgee family, U6), so the mock can neither observe nor drive any implementation of ours. Excluded for both sides by exact title (D-20261011-inquirer-prompts-editor-mock). |
| inquirer-prompts | packages/editor/editor.test.ts > editor prompt > clear value on failed validation | Asserts on, and answers through, vi.mock('@inquirer/external-editor')'s editAsync — a module of the incumbent's tree that caique/inquirer does not import (caique takes no dependency outside the burgee family, U6), so the mock can neither observe nor drive any implementation of ours. Excluded for both sides by exact title (D-20261011-inquirer-prompts-editor-mock). |
| inquirer-prompts | packages/editor/editor.test.ts > editor prompt > goes back to default value on failed validation | Asserts on, and answers through, vi.mock('@inquirer/external-editor')'s editAsync — a module of the incumbent's tree that caique/inquirer does not import (caique takes no dependency outside the burgee family, U6), so the mock can neither observe nor drive any implementation of ours. Excluded for both sides by exact title (D-20261011-inquirer-prompts-editor-mock). |
| inquirer-prompts | packages/editor/editor.test.ts > editor prompt > surfaces external-editor errors | Asserts on, and answers through, vi.mock('@inquirer/external-editor')'s editAsync — a module of the incumbent's tree that caique/inquirer does not import (caique takes no dependency outside the burgee family, U6), so the mock can neither observe nor drive any implementation of ours. Excluded for both sides by exact title (D-20261011-inquirer-prompts-editor-mock). |
| inquirer-prompts | packages/editor/editor.test.ts > editor prompt > displays custom waitingMessage | Asserts on, and answers through, vi.mock('@inquirer/external-editor')'s editAsync — a module of the incumbent's tree that caique/inquirer does not import (caique takes no dependency outside the burgee family, U6), so the mock can neither observe nor drive any implementation of ours. Excluded for both sides by exact title (D-20261011-inquirer-prompts-editor-mock). |
| inquirer-prompts | packages/editor/editor.test.ts > editor prompt > displays custom loadingMessage | Asserts on, and answers through, vi.mock('@inquirer/external-editor')'s editAsync — a module of the incumbent's tree that caique/inquirer does not import (caique takes no dependency outside the burgee family, U6), so the mock can neither observe nor drive any implementation of ours. Excluded for both sides by exact title (D-20261011-inquirer-prompts-editor-mock). |
| terminal-link | main | Sets supportsHyperlinks.stdout = true on the incumbent's detection module and expects OSC 8 on a headless runner. paratext takes no runtime dependency, so it cannot see an assignment to another package's module object; it answers from the same detection that module computes — graded environment by environment against supports-hyperlinks 4.5.0 in paratext/src/hyperlinks.test.ts (A27) — and on that runner the answer is no. The six cases that assign false stay graded and pass. |
| terminal-link | stderr | Sets supportsHyperlinks.stderr = true on the incumbent's detection module and expects OSC 8 on a headless runner. paratext takes no runtime dependency, so it cannot see an assignment to another package's module object; it answers from the same detection that module computes — graded environment by environment against supports-hyperlinks 4.5.0 in paratext/src/hyperlinks.test.ts (A27) — and on that runner the answer is no. The six cases that assign false stay graded and pass. |
| ink | build output files are not nested under build/src/ | build-output.ts grades ink's repository, not a package: its before hook deletes build/ and runs npm run build in the repo root, and its four cases assert the compiled layout (build/index.js, no build/src/), the exports paths of ink's own package.json, and the include of its tsconfig.json. There is no build script and no source tree in a vendored suite, so ink 8.0.0 itself fails all four here — and so would any implementation, because none of them is a behaviour of the library. |
| ink | package.json export paths resolve to existing files | build-output.ts grades ink's repository, not a package: its before hook deletes build/ and runs npm run build in the repo root, and its four cases assert the compiled layout (build/index.js, no build/src/), the exports paths of ink's own package.json, and the include of its tsconfig.json. There is no build script and no source tree in a vendored suite, so ink 8.0.0 itself fails all four here — and so would any implementation, because none of them is a behaviour of the library. |
| ink | build/index.js and build/index.d.ts exist | build-output.ts grades ink's repository, not a package: its before hook deletes build/ and runs npm run build in the repo root, and its four cases assert the compiled layout (build/index.js, no build/src/), the exports paths of ink's own package.json, and the include of its tsconfig.json. There is no build script and no source tree in a vendored suite, so ink 8.0.0 itself fails all four here — and so would any implementation, because none of them is a behaviour of the library. |
| ink | tsconfig.json include only contains src | build-output.ts grades ink's repository, not a package: its before hook deletes build/ and runs npm run build in the repo root, and its four cases assert the compiled layout (build/index.js, no build/src/), the exports paths of ink's own package.json, and the include of its tsconfig.json. There is no build script and no source tree in a vendored suite, so ink 8.0.0 itself fails all four here — and so would any implementation, because none of them is a behaviour of the library. |
| ink | Box width constraints only accept numbers | style-types.ts hands ink's own src/global.d.ts and fixtures/box-width-types.tsx to the TypeScript compiler and asserts no diagnostics. The vendored suite has no source tree and the shim has no types, so ink 8.0.0 itself fails it here (File 'src/global.d.ts' not found, Could not find a declaration file for module '../../shim.js'). What it checks — minWidth and maxWidth take numbers only, a string a type error — is held for controlroom/ink by @ts-expect-error lines in packages/controlroom/src/ink/types.test.ts, which the package's own typecheck compiles. |
| inkjs-ui | spinner › spinner | The case collects every write ink-testing-library sees across a spinner's life and expects ink 5's: on unmount ink 5 wrote nothing more in debug mode. ink 8 — the incumbent controlroom/ink follows since D-20261006-controlroom-ink-8 — ends a non-interactive debug run with a final \n (its own suite pins it: components.tsx › debug mode in CI keeps final newline separation after waitUntilExit) and then an empty write that resolves the exit only once every queued write has flushed. Measured 2026-10-06: @inkjs/ui 2.0.0's suite against real ink 8.0.0 on React 19.3.0 fails this case with the same diff ('\n' and '' after the last frame) and passes the other 102. The case grades which ink major @inkjs/ui is installed over, not the drop-in, so it is excluded on both runs rather than read as 102 here and 103 under ink 5. |
| ink-6 | background › Box background with hex color | The suite raises colour on its own chalk singleton (helpers/force-colors.ts, chalk.level = 3 before each case), which ink shares because ink colours through that same module. controlroom/ink colours through roundel, whose level is roundel's policy read from the environment and is nobody's singleton: under the oracle's FORCE_COLOR=true it is 1 on a laptop and 3 on GitHub Actions (supports-color's CI table, which roundel's policy reproduces), so the same case passed on one runner and failed on the other. Excluded so both runners publish one number; the drop-in's colour is graded by every other colour case in the suite, at the level the environment gives both implementations (D-20261005-controlroom-ink-drop-in). |
| ink-6 | background › Box background with rgb color | The suite raises colour on its own chalk singleton (helpers/force-colors.ts, chalk.level = 3 before each case), which ink shares because ink colours through that same module. controlroom/ink colours through roundel, whose level is roundel's policy read from the environment and is nobody's singleton: under the oracle's FORCE_COLOR=true it is 1 on a laptop and 3 on GitHub Actions (supports-color's CI table, which roundel's policy reproduces), so the same case passed on one runner and failed on the other. Excluded so both runners publish one number; the drop-in's colour is graded by every other colour case in the suite, at the level the environment gives both implementations (D-20261005-controlroom-ink-drop-in). |
| ink-6 | background › Box background with ansi256 color | The suite raises colour on its own chalk singleton (helpers/force-colors.ts, chalk.level = 3 before each case), which ink shares because ink colours through that same module. controlroom/ink colours through roundel, whose level is roundel's policy read from the environment and is nobody's singleton: under the oracle's FORCE_COLOR=true it is 1 on a laptop and 3 on GitHub Actions (supports-color's CI table, which roundel's policy reproduces), so the same case passed on one runner and failed on the other. Excluded so both runners publish one number; the drop-in's colour is graded by every other colour case in the suite, at the level the environment gives both implementations (D-20261005-controlroom-ink-drop-in). |
| ink-6 | background › Box background fills with hex color | The suite raises colour on its own chalk singleton (helpers/force-colors.ts, chalk.level = 3 before each case), which ink shares because ink colours through that same module. controlroom/ink colours through roundel, whose level is roundel's policy read from the environment and is nobody's singleton: under the oracle's FORCE_COLOR=true it is 1 on a laptop and 3 on GitHub Actions (supports-color's CI table, which roundel's policy reproduces), so the same case passed on one runner and failed on the other. Excluded so both runners publish one number; the drop-in's colour is graded by every other colour case in the suite, at the level the environment gives both implementations (D-20261005-controlroom-ink-drop-in). |
| ink-6 | background › Box background fills with rgb color | The suite raises colour on its own chalk singleton (helpers/force-colors.ts, chalk.level = 3 before each case), which ink shares because ink colours through that same module. controlroom/ink colours through roundel, whose level is roundel's policy read from the environment and is nobody's singleton: under the oracle's FORCE_COLOR=true it is 1 on a laptop and 3 on GitHub Actions (supports-color's CI table, which roundel's policy reproduces), so the same case passed on one runner and failed on the other. Excluded so both runners publish one number; the drop-in's colour is graded by every other colour case in the suite, at the level the environment gives both implementations (D-20261005-controlroom-ink-drop-in). |
| ink-6 | background › Box background fills with ansi256 color | The suite raises colour on its own chalk singleton (helpers/force-colors.ts, chalk.level = 3 before each case), which ink shares because ink colours through that same module. controlroom/ink colours through roundel, whose level is roundel's policy read from the environment and is nobody's singleton: under the oracle's FORCE_COLOR=true it is 1 on a laptop and 3 on GitHub Actions (supports-color's CI table, which roundel's policy reproduces), so the same case passed on one runner and failed on the other. Excluded so both runners publish one number; the drop-in's colour is graded by every other colour case in the suite, at the level the environment gives both implementations (D-20261005-controlroom-ink-drop-in). |
| ink-6 | background › Box background with hex color - concurrent | The suite raises colour on its own chalk singleton (helpers/force-colors.ts, chalk.level = 3 before each case), which ink shares because ink colours through that same module. controlroom/ink colours through roundel, whose level is roundel's policy read from the environment and is nobody's singleton: under the oracle's FORCE_COLOR=true it is 1 on a laptop and 3 on GitHub Actions (supports-color's CI table, which roundel's policy reproduces), so the same case passed on one runner and failed on the other. Excluded so both runners publish one number; the drop-in's colour is graded by every other colour case in the suite, at the level the environment gives both implementations (D-20261005-controlroom-ink-drop-in). |
| ink-6 | text › text with ansi256 color | chalk 5, which ink 6.8 uses, emits ansi256() unchanged at level 1 (\u001B[38;5;194m); chalk 6, which roundel/chalk is graded against, downsamples it to the nearest of the sixteen colours at level 1 (chalk.js asserts new Chalk({level: 1}).ansi256(196) is \u001B[91m). The drop-in colours through roundel, so at level 1 it follows chalk 6. The case passes wherever the level is 2 or more (GitHub Actions) and fails on a laptop at level 1, so it is excluded rather than left to read differently per runner (D-20261005-controlroom-ink-drop-in). |
| ink-6 | text › text with ansi256 background color | chalk 5, which ink 6.8 uses, emits ansi256() unchanged at level 1 (\u001B[38;5;194m); chalk 6, which roundel/chalk is graded against, downsamples it to the nearest of the sixteen colours at level 1 (chalk.js asserts new Chalk({level: 1}).ansi256(196) is \u001B[91m). The drop-in colours through roundel, so at level 1 it follows chalk 6. The case passes wherever the level is 2 or more (GitHub Actions) and fails on a laptop at level 1, so it is excluded rather than left to read differently per runner (D-20261005-controlroom-ink-drop-in). |
Control failures allowed
The control is the real host run through our harness. Where it fails a case for a reason that is the harness's and not the host's, the allowance is the exact count, and the gate refuses a different one — so it can neither hide a regression nor grow quietly.
| Host | Cases | Why |
|---|---|---|
| yargs | 2 | Real yargs reports its own version by reading the nearest package.json, and from inside a vendored copy that lookup finds ours. Two usage tests assert the version string; burgee scores 816 where yargs itself scores 814. Documented since the host was activated. |
| cosmiconfig | 1 | index.test.ts imports '../src/index.js' — cosmiconfig's own entry module, by path — in addition to the public entry, and vi.mocks ../src/Explorer and ../src/ExplorerSync to assert the CONSTRUCTOR ARGUMENTS the entry passes them. The vendor step generates a shim for every internal specifier the suite names, but ../src/index.js is the host's public entry reached by an internal path, so no shim is written and the file fails to load. That is one graded case, and it fails identically for the control and for the target: it is the harness's file-layout assumption, not a property of either implementation. This is the C4 shape seniority/spec.md finding 4 left open, decided here — the fix is in vendor.ts's internal-shim discovery, which is the harness lane's file, and until it lands the honest form is a named allowance of exactly one rather than an unexplained 240. |
| meow | 2 | Two cases no implementation can pass from a vendored copy of meow's tests, and they are one assertion made twice: build › main and test › return object each t.like the result against pkg: {name: 'meow'}. pkg is the nearest package.json, which is the vendored root, and that root is deliberately @vendored/meow-suite (rootPackage() — upstream's name would let Node's self-reference resolve meow to the root rather than the installed package). It carries upstream's description and version, which the suite also reads; the name is the one field it cannot. Re-measured 2026-09-27: the control is 146 / 148 and burgee/meow fails these two and nothing else. The version of this paragraph from 2026-09-21 named pkg normalization is lazy as the second and said build/index.js was absent; the control passes the first, and the shim serves the second. |
| signal-exit | 1 | signal-exit-test.ts > does not exit if user handles signal fails for signal-exit 4.1.0 itself. Its fixture, signal-listener.js, re-sends SIGTERM from a setTimeout inside the listener and expects the fourth to kill the process; on current Node the process exits cleanly after the first (calledListener=1, code=0, signal=null). Measured 2026-09-23 with the fixture requiring signal-exit directly — no shim — on Node 22.22, 24.13 and 26.10 on macOS, and on Node 24 in a Linux container. signal-exit's last release was 2023-07-29, before any of those Nodes. |
| ink | 1 | measure-element.tsx › calculate layout while rendering is throttled fails against ink 8.0.0 itself on darwin, three runs in three (2026-10-06): it renders, re-renders, waits 50 ms and asserts the last write is Width: 100, and ink's throttled trailing frame lands after that on this machine — the same program with a 300 ms wait writes Width: 100. A timing the case leaves to the machine, not a property of ink. controlroom/ink passes it. |
| yargs-17 | 1 | helpers > applyExtends > exposes applyExtends helper extends ./package.json from the cwd and asserts its name is yargs. The cwd is the vendored root, whose manifest is deliberately @vendored/yargs-17-suite (rootPackage() — upstream's name would let Node's self-reference resolve yargs to a file that is not here). The current-major row's two allowed failures are the same lookup reading the same file. |
Cases only some platforms register
The reference is the full suite on every platform, so a case a platform never registers is counted against us there rather than dropped.
| Host | Cases | Registered | Why |
|---|---|---|---|
| cosmiconfig | 2 | only on linux | search-strategies.test.ts wraps its global › finds config in OS default directory (XDG) describe in if (process.platform === 'linux'), so its two cases — async and sync — are not registered at all anywhere else. The suite is 241 cases on darwin and 243 on ubuntu, and that is the whole of the difference: no other conditional construct exists in the nine files (process.platform, os.platform, describe.each, skipIf and runIf return exactly this one hit). The reference is the ubuntu 243, because the reference has to be the full suite or the number means less on the machine that has more. Both pass against cosmiconfig and, since 2026-09-30, against seniority: its global search reads XDG_CONFIG_HOME (and APPDATA on Windows) as env-paths does, through runtime.ts (D-20260930-seniority-xdg-config-home). Until then it derived the directory from the home directory alone, failed both, and they were counted against it on every platform rather than excluded — excluding them would have raised the published rate from 76.5% to 77.2% by dropping two cases it lost. They are declared passing: 2 now for the same reason: a darwin run credits exactly the two it cannot register, and no more. |
| lilconfig | 2 | not on win32 | default for searchFrom till root directory and searches root directory correctly, each wrapped in if (process.platform !== 'win32') because both assert an fs call list rooted at /. They are the only conditional construct in the file — the isNodeV20orNewer branches pick an expected message and never add or drop a case, and this repository is Node 24 only. So the count does not move between darwin and ubuntu, which is why the reference is 77 on both and no number here changes. It is declared anyway: without it a Windows contributor's control registers 75 of 77 and goes red for doing exactly what the suite told it to, and finding that on their machine is worse than writing it here. |
| signal-exit | 8 | only on linux | all-integration-test.ts loops twice over signals (lines 31 and 64), and the list is the platform's: Linux adds SIGIO, SIGPOLL, SIGPWR and SIGSTKFLT, so the suite registers 135 cases on ubuntu and 127 on darwin — 4 signals × 2 loops. The reference is the ubuntu 135. All 8 pass against signal-exit and against closeout. Windows is not declared because it differs by more than one count can say (the first loop is empty there and the list is three signals long), and the ratchet runs on ubuntu. |
Directories vendored and not graded
Copied with the suite because its tests need them, and never run as tests themselves — or, where the reason says so, vendored whole and left out of the grade for what it costs to run.
| Host | Directory | Why |
|---|---|---|
| yargs | fixtures/ | Fixture programs the tests spawn as subprocesses. They match *.mjs only because that is how yargs writes an ESM fixture; running one as a test grades nothing. |
| yargs | helpers/ | utils.mjs, the output-capture helper the top-level tests import. A helper, never a test — mocha would load it and register no cases. |
| yargs | esm/ | yargs' separate ESM suite, which upstream runs under its own command. Found by this walk on 2026-09-09 and deliberately not graded yet: platform-shim-test.mjs imports lib/platform-shims/esm.mjs, an internal the vendor step's ../lib/ detector does not reach from a subdirectory, so grading the directory today adds three files that fail to load rather than three that measure anything. Named here so it is a decision with a date rather than a silence; grading it is its own piece of work. |
| execa | fixtures/ | The programs the tests spawn — noop.js, forever.js, ipc-echo.js and a hundred and forty more. Upstream's own glob skips the directory (ava did by convention before 10.1.0); running one as a test grades nothing, and several never exit. |
| execa | helpers/ | Modules the tests import (fixtures-directory.js, stdio.js, verbose.js …). Upstream's own glob skips the directory (ava did by convention before 10.1.0); each one declares no case of its own. |
| execa | convert/ | Vendored and not graded, for cost. execa's suite is 5,125 cases run one file at a time (upstream's own concurrency: 1, --test-concurrency=1 since 10.1.0), and the control measured 647 s across the twelve directories on 2026-09-27 — past the oracle's 300 s cap on one suite run, and more than half of the ratchet job's 20 minutes on its own. The graded slice is arguments/, methods/ and return/: how a command is built, the entry points, and the result and error a caller reads, which is the whole of what bellpull's run overlaps with execa (bellpull R1, R6). Nothing is hidden by leaving these out: graded over all 149 files the same day, bellpull passed 0 — every file imports execa, which bellpull does not export, and fails at link time. Re-measured at 10.1.0 on 2026-10-07 over all 152 files: 0, for the same reason. D-160. |
| execa | io/ | Vendored and not graded, for cost. execa's suite is 5,125 cases run one file at a time (upstream's own concurrency: 1, --test-concurrency=1 since 10.1.0), and the control measured 647 s across the twelve directories on 2026-09-27 — past the oracle's 300 s cap on one suite run, and more than half of the ratchet job's 20 minutes on its own. The graded slice is arguments/, methods/ and return/: how a command is built, the entry points, and the result and error a caller reads, which is the whole of what bellpull's run overlaps with execa (bellpull R1, R6). Nothing is hidden by leaving these out: graded over all 149 files the same day, bellpull passed 0 — every file imports execa, which bellpull does not export, and fails at link time. Re-measured at 10.1.0 on 2026-10-07 over all 152 files: 0, for the same reason. D-160. |
| execa | ipc/ | Vendored and not graded, for cost. execa's suite is 5,125 cases run one file at a time (upstream's own concurrency: 1, --test-concurrency=1 since 10.1.0), and the control measured 647 s across the twelve directories on 2026-09-27 — past the oracle's 300 s cap on one suite run, and more than half of the ratchet job's 20 minutes on its own. The graded slice is arguments/, methods/ and return/: how a command is built, the entry points, and the result and error a caller reads, which is the whole of what bellpull's run overlaps with execa (bellpull R1, R6). Nothing is hidden by leaving these out: graded over all 149 files the same day, bellpull passed 0 — every file imports execa, which bellpull does not export, and fails at link time. Re-measured at 10.1.0 on 2026-10-07 over all 152 files: 0, for the same reason. D-160. |
| execa | pipe/ | Vendored and not graded, for cost. execa's suite is 5,125 cases run one file at a time (upstream's own concurrency: 1, --test-concurrency=1 since 10.1.0), and the control measured 647 s across the twelve directories on 2026-09-27 — past the oracle's 300 s cap on one suite run, and more than half of the ratchet job's 20 minutes on its own. The graded slice is arguments/, methods/ and return/: how a command is built, the entry points, and the result and error a caller reads, which is the whole of what bellpull's run overlaps with execa (bellpull R1, R6). Nothing is hidden by leaving these out: graded over all 149 files the same day, bellpull passed 0 — every file imports execa, which bellpull does not export, and fails at link time. Re-measured at 10.1.0 on 2026-10-07 over all 152 files: 0, for the same reason. D-160. |
| execa | resolve/ | Vendored and not graded, for cost. execa's suite is 5,125 cases run one file at a time (upstream's own concurrency: 1, --test-concurrency=1 since 10.1.0), and the control measured 647 s across the twelve directories on 2026-09-27 — past the oracle's 300 s cap on one suite run, and more than half of the ratchet job's 20 minutes on its own. The graded slice is arguments/, methods/ and return/: how a command is built, the entry points, and the result and error a caller reads, which is the whole of what bellpull's run overlaps with execa (bellpull R1, R6). Nothing is hidden by leaving these out: graded over all 149 files the same day, bellpull passed 0 — every file imports execa, which bellpull does not export, and fails at link time. Re-measured at 10.1.0 on 2026-10-07 over all 152 files: 0, for the same reason. D-160. |
| execa | stdio/ | Vendored and not graded, for cost. execa's suite is 5,125 cases run one file at a time (upstream's own concurrency: 1, --test-concurrency=1 since 10.1.0), and the control measured 647 s across the twelve directories on 2026-09-27 — past the oracle's 300 s cap on one suite run, and more than half of the ratchet job's 20 minutes on its own. The graded slice is arguments/, methods/ and return/: how a command is built, the entry points, and the result and error a caller reads, which is the whole of what bellpull's run overlaps with execa (bellpull R1, R6). Nothing is hidden by leaving these out: graded over all 149 files the same day, bellpull passed 0 — every file imports execa, which bellpull does not export, and fails at link time. Re-measured at 10.1.0 on 2026-10-07 over all 152 files: 0, for the same reason. D-160. |
| execa | terminate/ | Vendored and not graded, for cost. execa's suite is 5,125 cases run one file at a time (upstream's own concurrency: 1, --test-concurrency=1 since 10.1.0), and the control measured 647 s across the twelve directories on 2026-09-27 — past the oracle's 300 s cap on one suite run, and more than half of the ratchet job's 20 minutes on its own. The graded slice is arguments/, methods/ and return/: how a command is built, the entry points, and the result and error a caller reads, which is the whole of what bellpull's run overlaps with execa (bellpull R1, R6). Nothing is hidden by leaving these out: graded over all 149 files the same day, bellpull passed 0 — every file imports execa, which bellpull does not export, and fails at link time. Re-measured at 10.1.0 on 2026-10-07 over all 152 files: 0, for the same reason. D-160. |
| execa | transform/ | Vendored and not graded, for cost. execa's suite is 5,125 cases run one file at a time (upstream's own concurrency: 1, --test-concurrency=1 since 10.1.0), and the control measured 647 s across the twelve directories on 2026-09-27 — past the oracle's 300 s cap on one suite run, and more than half of the ratchet job's 20 minutes on its own. The graded slice is arguments/, methods/ and return/: how a command is built, the entry points, and the result and error a caller reads, which is the whole of what bellpull's run overlaps with execa (bellpull R1, R6). Nothing is hidden by leaving these out: graded over all 149 files the same day, bellpull passed 0 — every file imports execa, which bellpull does not export, and fails at link time. Re-measured at 10.1.0 on 2026-10-07 over all 152 files: 0, for the same reason. D-160. |
| execa | verbose/ | Vendored and not graded, for cost. execa's suite is 5,125 cases run one file at a time (upstream's own concurrency: 1, --test-concurrency=1 since 10.1.0), and the control measured 647 s across the twelve directories on 2026-09-27 — past the oracle's 300 s cap on one suite run, and more than half of the ratchet job's 20 minutes on its own. The graded slice is arguments/, methods/ and return/: how a command is built, the entry points, and the result and error a caller reads, which is the whole of what bellpull's run overlaps with execa (bellpull R1, R6). Nothing is hidden by leaving these out: graded over all 149 files the same day, bellpull passed 0 — every file imports execa, which bellpull does not export, and fails at link time. Re-measured at 10.1.0 on 2026-10-07 over all 152 files: 0, for the same reason. D-160. |
| meow | fixtures/ | Twenty-four CLI programs the tests spawn with execa — two thirds of everything under test/. ava's own globs exclude them and the harness's walk did not, so ava was handed them as test files and each one that exits non-zero was counted: the row read # fail 22 beside # tests 148 / # pass 144, a summary that disagrees with itself. Ungraded, test/ is 12 test files. |
| signal-exit | fixtures/ | Programs the tests spawn as child processes to watch them exit — exit.js, sigint.js, sigkill.js and eighteen more. They match *.{js,ts} because that is simply how they are written, and running one as a test grades nothing: it installs a handler and kills itself. Two of them (signal-capture.js, sigkill.js) reach the implementation through ../../dist/, which is why dist/cjs/signals.js is on the internal-shim list at all. |
| ink | helpers/ | Ten modules the tests import — render-to-string.ts, create-stdout.ts, create-stdin.ts, test-renderer.ts, force-colors.ts, act.ts, mock-timer-calls.ts, reconstruct-terminal.ts, and run.ts and term.ts, which spawn a fixture under node-pty. Upstream's npm test globs test/*.{ts,tsx}, one level, so none of them is run as a test; none declares a case. |
| ink | fixtures/ | Sixty TSX programs run.ts and term.ts spawn in a pseudo-terminal (node --import=tsx fixtures/<name>.tsx) for the cases that need a real TTY, and the one style-types.ts hands to the TypeScript compiler. Upstream's npm test globs test/*.{ts,tsx}, one level, so none is run as a test; each renders an app and exits, and grades nothing on its own. |
| yargs-17 | helpers/ | utils.cjs, the output-capture helper the top-level tests require. A helper, never a test. |
| ink-6 | helpers/ | Seven modules the tests import — render-to-string.ts, create-stdout.ts, create-stdin.ts, test-renderer.ts, force-colors.ts, and run.ts and term.ts, which spawn a fixture under node-pty. Upstream's own ava.files excludes the directory (!test/helpers/**/*); none of them declares a case. |
| ink-6 | fixtures/ | Twenty-eight TSX programs run.ts and term.ts spawn in a pseudo-terminal (node --import=tsx fixtures/<name>.tsx) for the cases that need a real TTY: exit, raw mode, useInput keystrokes, erase and <Static> in a short viewport. Upstream's ava.files excludes them (!test/fixtures/**/*); each renders an app and exits, and grades nothing on its own. |
Graded as a ceiling, not a drop-in
The incumbent's own suite, run against a package that deliberately does not implement the
incumbent's API. The rate is the distance between the two, published so it is measured rather
than asserted — it is not a façade on its way to 100%, and burgee migrate neither rewrites to
it nor reports it as a path.
| Host | Graded against | Why |
|---|---|---|
| execa | bellpull | bellpull ships no execa API and will not: run resolves a record on a non-zero exit where execa throws, so an execa façade would silently stop every caller's error path firing (bellpull spec R7). The row is graded to publish that distance as a number rather than as a sentence — every graded file imports execa, which bellpull does not export, and fails at link time. |
Supported majors
A major is claimed only where the host's own suite at that major grades the front-end
level with the host itself — every case the host passes in this harness, passed. The current
major is the release the table above grades. An older major is its own suite, vendored at that
major's last tag and graded in CI beside the current one; the claim is
SUPPORTED_MAJORS in burgee/src/compat.ts, which burgee migrate reads, and a lock holds it
to these rows (C1).
| Host | Majors claimed | Release graded | Older majors graded |
|---|---|---|---|
| commander | 14, 15 | 15.0.0 | commander-14 (14.0.3) — claimed |
| extra-typings | 15 | 15.0.0 | — |
| yargs | 17, 18 | 18.2.0 | yargs-17 (17.7.3) — claimed |
| chalk | 6 | 6.0.1 | — |
| picocolors | 1 | 1.1.1 | — |
| supports-color | 11 | 11.0.0 | — |
| yoctocolors | 2 | 2.2.0 | — |
| ora | 9 | 9.4.1 | — |
| log-update | 8 | 8.0.0 | — |
| boxen | 9 | 9.0.0 | — |
| cli-table3 | 0 | 0.6.5 | — |
| string-width | 8 | 8.3.0 | — |
| strip-ansi | 7 | 7.2.0 | — |
| cross-spawn | 7 | 7.0.6 | — |
| which | 7 | 7.0.0 | — |
| execa | — | — | — |
| rc | 1 | 1.2.8 | — |
| wrap-ansi | 10 | 10.0.2 | — |
| slice-ansi | 9 | 9.0.1 | slice-ansi-7 (7.1.2) — graded, not claimed |
| cli-truncate | 6 | 6.1.1 | — |
| widest-line | 6 | 6.0.0 | — |
| cosmiconfig | 10 | 10.0.1 | — |
| lilconfig | 3 | 3.1.4 | — |
| dotenv | 18 | 18.0.8 | dotenv-17 (17.4.2) — graded, not claimed |
| find-up | 8 | 8.0.0 | — |
| clack | 1 | 1.8.1 | — |
| inquirer-prompts | 8 | 8.7.3 | — |
| inquirer-core | 12 | 12.0.4 | — |
| meow | 14 | 14.1.0 | — |
| ansi-escapes | 7 | 7.3.0 | — |
| terminal-link | 5 | 5.0.0 | — |
| term-img | 7 | 7.1.0 | — |
| restore-cursor | 5 | 5.1.0 | — |
| exit-hook | 5 | 5.1.0 | — |
| signal-exit | 4 | 4.1.0 | — |
| onetime | 8 | 8.0.0 | — |
| cli-cursor | 5 | 5.0.0 | — |
| ink | 8 | 8.0.0 | ink-6 (6.8.0) — graded, not claimed |
| inkjs-ui | — | — | — |
| Host | Front-end | burgee | rate | control | rate | internals (burgee) | internals (control) |
|---|---|---|---|---|---|---|---|
| commander-14 | burgee/commander/v14 | 1331 / 1331 | 100.0% | 1331 / 1331 | 100.0% | 25 / 36 | 36 / 36 |
| yargs-17 | burgee/yargs/v17 | 793 / 794 | 99.9% | 793 / 794 | 99.9% | 23 / 23 | 23 / 23 |
| slice-ansi-7 | linegauge/slice | 14 / 15 (1 the host marks failing and we pass) | 93.3% | 15 / 15 | 100.0% | — | — |
| dotenv-17 | seniority | 107 / 141 | 75.9% | 141 / 141 | 100.0% | 0 / 0 | 7 / 13 |
| ink-6 | controlroom/ink | 540 / 584 | 92.5% | 584 / 584 | 100.0% | 0 / 0 | 148 / 148 |
An older major that grades below level is published here and not claimed: a program on it is
left alone by burgee migrate, and the note in hosts.ts names the cases that differ.
A front-end does not reach 1.0 until its rate is 100% (C7). Below that it ships pre-1.0 and is never described as compatible. The rate ratchets: a pull request that lowers it fails CI unless it also lowers the committed baseline, an edit a reviewer sees (C5).
Other hosts
| Host | Status | Why |
|---|---|---|
| cac | planned | Deferred: 49.4M/wk is almost entirely vite and vitest bundling it, not developers choosing it, so a façade converts few people. |
| citty | planned | Deferred for the same reason as cac: nitro and unjs bundle it. |
| oclif | rejected | Its API is inseparable from its shape: a project layout, a build step and a generator. A façade could not be adopted without adopting the shape that Z1 exists to prevent, and oclif does 10.9M/wk against commander’s 508M, so the shape is also what lost. |
Reproduce
npm run compat -- --vendor --control # vendor both suites, grade the real hosts
npm run compat # grade burgee
npm run compat -- --majors --control # the older majors, against the real hosts (C1)
npm run compat -- --majors # the older majors, against burgee