What the instrument measured.
10 / 2 legibility
8 / 3 constraint
6 / 8 interaction
Measured against a pinned commit.
- Repository
- facebook/astryx
- Revision
- 53bc34db7c9d, committed 2026-09-09
- System directory
- packages/core (@astryxdesign/core)
- Components found
- 276, of which 270 carry documentation and 174 have a colocated doc artifact
- Agent context
- 2 context files, 0 skills
- Tokens
- 237 distinct, across 11 sources (11 authored); dark mode generated
- Adherence surface
- 2,618 files outside the system, graded own-docs-site: the apps, and the lab, charts and richtext packages.
What the counts point at.
Each finding carries its count, its severity and up to three file references. A finding on a property that was not banded is still a finding; it is reported without a score.
The design system's own component source hardcodes colour and size values instead of referencing its own tokens. This happens in 885 of 3595 total colour/size references in system source (24.6%), so an agent copying that file as a pattern copies the hardcoding along with it.
- packages/core/src/theme/syntax/presets.ts:23 #c678dd
- packages/core/src/theme/domainTokens/dataTokens.ts:21 #0171E3
- packages/core/src/Spinner/Spinner.doc.mjs:14 10px
Design tokens are declared in more than one place in this repository's own authored source, so a value changed in one must be kept in agreement with the others by hand.
- packages/core/src/theme/tokens.stylex.ts stylex, 162 token(s)
- packages/lab/src/SVGIcon/tokens.stylex.ts stylex, 24 token(s)
- internal/vibe-tests/fixtures/shadcn-tailwind-v4-established/src/index.css css, 19 token(s)
Some components carry no documentation an agent could find. That is 6 of the 276 components found in system source, with no head comment on their own source, no colocated doc artifact matching their name, and no docs/ entry, so an agent must reverse-engineer their contract from implementation alone.
- packages/core/src/Table/plugins/filtering/index.ts
- packages/core/src/Table/plugins/pagination/index.ts
- packages/core/src/Table/plugins/selection/index.ts
The application hardcodes colour values instead of using the design system's own tokens, so an agent editing nearby code has no signal that a token exists and will likely repeat the pattern.
- packages/themes/probe/src/probeTheme.ts:39 hsl(19.9 84% 57%)
- packages/cli/assets/templates/themes/butter/butterTheme.ts:29 #52237b
- packages/themes/butter/src/butterTheme.ts:29 #52237b
The application defines its own component under the same name as one the design system already ships. This happens for 36 of the 308 component names the design system exports, so an agent importing that name by convention can silently get the wrong implementation.
- apps/docsite/src/app/playground/defaultCode.ts DEFAULT
- apps/docsite/src/app/playground/themeEditor/constants.ts ALL
- apps/example-vite-tailwind/src/App.tsx:24 App
Some screens render a raw HTML element in place of the system component built for it. This happens in 69 of 1227 total uses of a replaceable element (5.6%), so its styling and behaviour can drift independently of the system.
- apps/sandbox/src/components/themePreview/ThemeAuditDrawer.tsx:512 <button>
- internal/vibe-tests/fixtures/enterprise-scoped-synthetic/src/App.tsx:81 <button>
- internal/vibe-tests/fixtures/shadcn-tailwind-v4-established/src/App.tsx:114 <input>
Built twice: cold, then informed.
Set up from its own documentation, in full, with every step that could not be followed recorded. Draw 1 of 3, judged; draws 2 and 3 pending.




What setting it up found
The README's own top callout reads: '**Building with an AI agent?** Run `npx astryx init` first.' Run literally as written (bare `npx astryx`, before @astryxdesign/cli is a project dependency), this resolves to a completely different, unaffiliated public npm package named `astryx` (verified on the public npm registry: a single unrelated maintainer, unpacked size 424 bytes, no `bin` entry at all, so it is not even a runnable CLI). The correct scoped form, `npx @astryxdesign/cli init`, appears only later, inside the CLI's own runtime output (the redirect banner printed by docs.mjs), never at the point in the README where the bare instruction is given. This recipe does not run either form; see the last entry below for why.
The README's 'Component Docs' section instructs exactly this for full component documentation: `node node_modules/@astryxdesign/core/docs.mjs Button`. Running that command exactly as documented, immediately after installing the package per the Quick Start, does not print component docs. It always prints a fixed redirect banner instructing you to install and run the CLI instead, regardless of whether @astryxdesign/cli happens to be installed. The documented command never succeeds at its own stated purpose.
The 'Vite' quick-start section is two lines: the install command, then 'Same CSS imports and providers as above. No build plugins needed; Astryx ships pre-built.' The providers.tsx example given 'above' (in the immediately preceding Next.js section) imports `Link from 'next/link'` and opens with `'use client'`, both Next.js-specific and absent from a Vite project (there is no next/link module and no server/client component split to declare). Followed literally, the Vite section point at code that cannot compile outside Next.js. In practice, Astryx's Link component falls back to a plain `<a>` element when no LinkProvider is supplied (confirmed by reading the Link source directly (node_modules/@astryxdesign/core/src/Link/: LinkContext.ts defaults the context to null and useLinkComponent.ts resolves `as ?? ctx?.component ?? 'a'`, so with no provider it falls back to a native anchor); the README never states this default exists), so this recipe's providers.tsx omits LinkProvider entirely for the router-less Vite scaffold. That fallback behavior was filled in from source, not from documentation, because the docs are silent on it.
@astryxdesign/core's package.json declares `@stylexjs/stylex: ^0.19.0` as a required peerDependency, but the Quick Start's pre-built CSS/JS path (the one this recipe follows, for both Next.js and Vite, explicitly billed as needing 'no build plugins, no PostCSS, no Babel config') never mentions installing it and never uses it at runtime (grepped dist/index.js for every occurrence of 'stylex': none). Modern npm (7 and later, used here) auto-installs unmet peer dependencies by default, so @stylexjs/stylex 0.19.0 was silently pulled into node_modules anyway with no error; an npm configuration that does not auto-install peers, or an older npm major version, would instead surface an ERESOLVE-style peer-dependency warning about a package the documented install path does not actually need.
Screens generated
| Arm | Draw | Date | Duration | Files read | Outcome |
|---|---|---|---|---|---|
| cold | 1 | 2026-08-14 | 12 min | 2 | completed |
| informed | 1 | 2026-08-14 | 12 min | 7 | completed |
Model: claude-opus-4-8. Generated under GitHub Actions isolation, the agent's own environment, with the frozen task text and no interaction. Files read is the number of repository files the agent opened, from its own transcript.
Cold arm, draw 1 · the working
- Counted
- 3 hardcoded colours, 26 hardcoded dimensions, 0 raw elements where a component exists, 67 uses of the system's components, 0 foreign UI imports, 0 invented names.
- Read
- 0 documentation reads, 18 reads of the package's source or types; went looking for documentation unprompted: no.
- Checklist
- I1 yes · I2 yes · I3 yes · I4 yes · I5 yes · I6 yes · I7 yes · I8 no · I9 yes · I10 noI8: loading and load failure render, not observed (probe: loading state seen at first paint; load-failure state not rendered under the query spellings tried). I10: every control keyboard reachable, not observed (probe tab walk: 13 toggles, 0 time inputs, 1 selects reached, save not reached, focus visible on 100%)
- Composition
- Six no's: conforms. Every group on a Card surface, page inset on all sides, ~90px of ground below the last card at 1280. Matches the 2026-08-14 read (preferred over informed).
Informed arm, draw 1 · the working
- Counted
- 0 hardcoded colours, 1 hardcoded dimensions, 0 raw elements where a component exists, 118 uses of the system's components, 0 foreign UI imports, 0 invented names.
- Read
- 0 documentation reads, 6 reads of the package's source or types.
- Checklist
- I1 yes · I2 yes · I3 yes · I4 yes · I5 no · I6 yes · I7 no · I8 no · I9 yes · I10 noI5: unsaved changes indicated, not observed (probe: changed a control; unsaved wording did not appear). I7: success and failure each communicated, not observed (probe: success wording seen; failure wording not seen (demo control "Make the next save fail")). I8: loading and load failure render, not observed (probe: loading state seen at first paint; load-failure state not rendered under the query spellings tried). I10: every control keyboard reachable, not observed (probe tab walk: 15 toggles, 0 time inputs, 1 selects reached, save not reached, focus visible on 100%)
- Composition
- Q1: content touches a viewport edge; Q3: a unit sits on no surface. the 2026-08-14 read, reproduced: groups sit in undifferentiated flow separated by rules (Q3), and the demo-controls panel runs wider than the content column and ends within ~12px of the page bottom (Q1). The informed agent used the system's layout frame (VStack, HStack, Grid) and no card.
Judged 2026-09-09 by an agent under authorisation; a second read is pending. Constraint from the counts, banded twice, on how many kinds of the system's rules the output broke and on how many times it broke them, with the worse of the two published (deviation of 2026-09-10). Legibility and completeness from the transcript and the source, interaction from the ten-item checklist, composition from the six contract questions. Hover a checklist item for its observation.
Clone the repository at the revision above, point the instrument at the system directory, and the same numbers come out. The field manifest, the run script and the full payload this page was built from are committed beside it. Methods.