Benchmarks Studio · suite packs

Small, versioned claims you can re-run.

A pack is data: it names deterministic checks that already exist in this repository and points at the committed results file that contains them. It cannot add a provider, a credential or a spend. The validator refuses a pack that claims a pass the results file does not contain.

The committed packs

Evidence is read from benchmarks/results/latest.json, generated 2026-11-15 13:30 UTC from source951b313. A recorded pack is a dated observation, not a live claim about this exact checkout.

CF-COMPOSE-01 · cf-compose-01.json

Cloudflare composition: local evidence foundation

Not run

Reuses all eleven sandbox save/reopen and blueprint round-trip checks, then exercises the real read-only provider operations view with fictional scope, coverage, dependency and cost observations. It establishes local contracts, not live Cloudflare capability.

Provider free
Yes
Network
Not used
Credentials
None
Spend
$0
Checks
15 deterministic
Evidence
none yet

foundation

  • sandbox-save-reopen-edit-survives-reopen

    Not present in the committed results file.

  • sandbox-save-reopen-fake-save-detected

    Not present in the committed results file.

  • sandbox-save-reopen-tampered-save-rejected

    Not present in the committed results file.

  • sandbox-save-reopen-approve-makes-public

    Not present in the committed results file.

  • sandbox-save-reopen-hide-removes-publicly

    Not present in the committed results file.

  • sandbox-save-reopen-restore-keeps-moderation

    Not present in the committed results file.

  • sandbox-save-reopen-export-still-reads-restored-state

    Not present in the committed results file.

  • blueprint-roundtrip-export-succeeds

    Not present in the committed results file.

  • blueprint-roundtrip-fields-survive

    Not present in the committed results file.

  • blueprint-roundtrip-preview-only-reported

    Not present in the committed results file.

  • blueprint-roundtrip-invalid-refused

    Not present in the committed results file.

  • provider-scope-mismatch-refused

    Not present in the committed results file.

  • provider-incomplete-observations-stay-unknown

    Not present in the committed results file.

  • provider-explicit-dependencies-preserved

    Not present in the committed results file.

  • provider-unknown-cost-no-retirement-authority

    Not present in the committed results file.

Not run. This pack has no passing evidence in the committed file, so it is not shown as a result. No committed baseline is attached to this pack. Retain each source-pinned local output separately; fixture results do not establish completed Cloudflare journeys.

  • A local pass covers the named source functions and fictional observations only; it is not a browser, deployed application or live Cloudflare check.
  • Provider operations are derived from supplied observations. No account is contacted, and no actual resource permissions, provisioning or preview isolation are established.
  • Only observed service/D1/R2/KV bindings are represented. Queues, Workflows, external clients and unread Worker settings remain outside the known dependency graph.
  • This pack does not measure model quality, production latency, upload throughput, provider charges, data restoration or completion of the Cloudflare wishlist.
  • The zero-spend requirement prohibits provider execution here; it does not turn unknown cloud resource charges into zero costs or grant retirement authority.
  • Run this optional pack explicitly through the existing runner; its separate directory does not add a blanket check to routine releases.

FLOW-01 · flow-01.json

Sandbox flow: save, reopen and export a blueprint

Recorded evidence

A provider-free pack over the public sandbox's deterministic store, moderation and blueprint checks. It proves the same edits survive a save and reopen, a tampered save is refused, a hidden note stays hidden after restore, and a restored sandbox still exports a valid blueprint.

Provider free
Yes
Network
Not used
Credentials
None
Spend
$0
Checks
6 deterministic
Evidence
benchmarks/results/latest.json

sandbox-save-reopen

  • edit-survives-reopen

    7 edits (name, tagline, theme, accent, layout, block, link) reopened byte-identically after JSON.stringify/parse.

  • tampered-save-rejected

    A save with no contributions and a save with version 2 were both refused; each recovered to the initial sandbox instead of half-applying.

  • restore-keeps-moderation

    Version 1 restored the saved name while note-1 stayed hidden and note-2 stayed pending; appearance and moderation travel separately.

  • export-still-reads-restored-state

    Export after restore produced preset “composition” with 2 links.

blueprint-roundtrip

  • export-succeeds

    Exported version 2 preset “composition” with 3 links.

  • fields-survive

    Name, introduction, theme, layout, both enabled blocks and 3 links survived export → JSON → re-import.

Evidence recorded 2026-11-15 13:30 UTC from source951b31384a6abb85f947fbc46c87a3ee95363c63. The pack checks the same ids in the committed file.

  • The pack runs only the deterministic checks named here. It does not measure model quality, latency or cost.
  • A pass means these in-repo helpers held on the committed source; it is not a provider, deployment or browser check.
  • The evidence is a dated run. A later source revision can make the result stale, and the page says so rather than assuming it still holds.

OBS-01 · obs-01.json

Offline observability fixture

Not run

Check the relationship between a failed request, its spans and a log. The fictional fixture includes a later recovery, an independent control and missing, duplicate and unsupported signals.

Provider free
Yes
Network
Not used
Credentials
None
Spend
$0
Checks
4 deterministic
Evidence
none yet

obs-01

  • observed-correlated-failure

    Not present in the committed results file.

  • observed-independent-recovery

    Not present in the committed results file.

  • duplicate-or-missing-unproven

    Not present in the committed results file.

  • unsupported-signal-unproven

    Not present in the committed results file.

Not run. This pack has no passing evidence in the committed file, so it is not shown as a result. The catalogue has no committed run receipt. Run the offline fixture and open its JSON result below.

  • These are fictional request, log and trace records. Their timings come from the fixture timestamps.
  • The checks run locally with zero provider use or spend. Live telemetry delivery, incident diagnosis and remediation require separate evidence.

Can App Composer build App Composer?

The challenge is to rebuild and operate App Composer and its supported apps in an independently owned Cloudflare account, then repeat the process from the rebuilt instance. If a creator can reproduce this system from documented inputs, they can use the same method to understand, own and evolve their own projects.

Target
CF-R08 · v1
Status
Not run
Evidence entries
0
Owner
Self-rebuild challenge

The A → B → C sequence

  1. Pin A. Freeze a baseline and publish the permitted blueprints and contracts.
  2. Build B. Use a supported, authorised agent in a separate account with its own credentials. Use only the published inputs and record any assistance.
  3. Repeat from B. Create and evolve an app using B, then use B to construct C. Compare both rebuilds with the frozen parity checklist and record any gaps.

First milestone · A → B

Deploy a functional Composer slice in a separately owned account and exercise real auth, data, media and operations. B → C starts after that evidence is available. Routine releases continue independently of this challenge.

Before a run

Run one pack

From a checkout of this repository, run only the checks the pack names:

node benchmarks/runner.mjs --suite FLOW-01

The optional CF-COMPOSE-01 pack lives in a separate directory, so name it explicitly:

node benchmarks/runner.mjs --suites benchmarks/cloudflare-composition --suite CF-COMPOSE-01

The runner validates the pack, resolves the named modules, runs only those checks and prints the same result shape as a full run. Without --suite, the existing .mjs suite discovery is unchanged.

Inspect a local run result

Open a runner result here to inspect it in this tab.

node benchmarks/runner.mjs --suites benchmarks/cloudflare-composition --suite CF-COMPOSE-01 > cf-compose-01-result.json

Run this command from a checkout of this repository, then open the JSON below.

Private run history

Keep one result in this browser if you want to reopen it later. Anyone using this browser profile can open it.

Inspect a suite definition

Open a versioned JSON suite to check its format and see which references appear in the current catalogue. The file stays in this tab.

Create a suite definition

Choose existing checks from one catalogue pack. Download the definition or prepare it for a project.

Provider-free checks

Project suites

Choose an owner workspace and a project to keep a definition. Saved definitions are private to the owner who saved them.

Keep the suite JSON and its Markdown run task together when you hand the check to your coding agent.

Open or create a suite definition and choose Use in project suites.

Private run history

Signed-in workspace owners can save a report here to reopen on another device.

What a pack does not prove