GTM Autoresearch already scores a client's container on 12 dimensions, mutates it, and keeps or reverts each change overnight. This box adds the second loop from the Karpathy-loop playbook: it reads every run's ledger, names the habits that keep failing, and proposes a rewrite of how the loop works. The scorer and the rules stay locked, and the owner still publishes.
The run checks the code against two references: the GTM AutoResearch field guide (what the loop is supposed to do) and the Jev container guide (how a winner should be judged). Everything here comes from the run in the stream above: arb check all and arb autoloop on main at 62dfaa3, reading the 7 recorded run ledgers. Every score was produced by the repo's own scorer, and every audit finding by the Jev container guide's own scripts. The box reimplements neither, and it called no model, ad platform or GTM API.
| Check | Result | What it found |
|---|
After round 5 the only error left in BLADE's container was CE - GA4 Ecom Events: a regex trigger matching view_item_list|select_item|view_cart|add_payment_info, flagged for lacking an EQUALS filter. pickStrategy() always picks the lowest-scoring dimension with an error, and the scores never changed, so it picked trigger quality 24 times. A real fix splits the trigger into 4 EQUALS triggers and re-points 2 tags: 6 entity changes against a budget of 3. Every attempt broke something and was reverted. Meanwhile folder organization (5.2pp of weighted gap), CAPI coverage (4.0pp) and naming (3.9pp) were never touched. Trigger quality was gap #7 of 12.
Source: the AutoLoop walkthrough ingested with TwelveLabs (YouTube qLfSDQ5NGh0). Left is what the video names; right is where it lives in gtm-autoresearch, with the box's check result.
| In the video | In gtm-autoresearch | Check |
|---|
The field guide already maps the video in chapter 01 and names the meta layer: it "may not touch the scorer or the Jev gates." The box takes each chapter's checkable claims and runs them. Shadow stays shadow: where the guide calls a rule a future gate, the check reports "not built", never "fail".
| Chapter | The guide says | Checks |
|---|
Reverts and the regression stop. Chapters 01 and 04 say a revert "increments the regression counter" and MAX_REGRESSIONS is "3 consecutive reverts". In run-gtm-loop.ts, regressionCount only moves when the working score drops, and a revert never drops it. That is why BLADE ran 24 reverts in a row. program.md and the prompt. Chapter 01 says the strategy order and constraints from program.md become the mutation prompt. parseProgram() reads only file paths; both lists are hardcoded. The first run. Chapter 04 records it as 84.3% → 91.2%, "2 kept, 3 reverted". The ledger 2026-04-07T231552.json shows 3 kept, 0 reverted and 2 rounds with no response from the mutation provider. Fix the code for the first two and the table for the third.
One CLI, one Worker, one D1 table, all inside the repo under box/. The GTM loop keeps running where it runs today, on the Mac with the Claude CLI and the MCP-fed ads snapshot. The box reads what it leaves behind.
MAC (inner loop, unchanged) OWNER /refresh-ads-snapshot blade Meta · Google Ads · GTM via MCP npx tsx scripts/run-gtm-loop.ts content/gtm-templates/BLADE/program.md score ─▶ pickStrategy ─▶ Claude mutates ≤3 entities ─▶ validate ─▶ keep / revert │ evals/eval_gtm_signal_quality.ts (lock: .claude deny evals/**) ▼ winning/*.json · loop-results/*.json (the ledger) · data audit · staging workspace ──▶ publishes │ ▼ git push / Worker cron ─▶ workflow_dispatch box.yml GitHub Actions arb check all G0 contract · G1 locked scorer · G2 ledger · G3 AutoLoop · G4 beyond the score arb autoloop read ledgers ▸ find habits ▸ rewrite "Edit Strategy" only ▸ stage for review │ POST /api/events (Bearer ARB_INGEST_TOKEN) ▼ ┌──────────────────────── CLOUDFLARE ─────────────────────────────────┐ │ autoresearch-in-a-box Worker │ │ Assets / guide · /run/ run view │ │ /api/events ──▶ D1 events (run_id, seq, kind, body) │ │ /api/stream ◀── SSE, resumes from Last-Event-ID ──▶ any browser │ │ cron 08:23 UTC ──▶ dispatch box.yml │ │ jev-gateway (shadow judge) · jev-site-scan (observed vs configured)│ └─────────────────────────────────────────────────────────────────────┘ │ habit found ▼ PR: program.md "Edit Strategy" + evidence rounds ──▶ owner approves ──▶ next run starts with it
Check IDs are code in box/checks/index.mjs. Unbuilt work reports "not built" instead of passing. Chip colors are the result from the recorded run; hover a chip for its reason.
The video's warning is that a feature can pass every check and still be incomplete. Here the second opinion already exists: the Jev container guide ships a deterministic reader and audit. The box runs those scripts, copied unchanged into box/jev/, on each client's seed and best winner. No model was called.
| Guide step | Owner | In the loop today | On BLADE, seed → winner |
|---|---|---|---|
| 1 · Parse + validate | code | validateContainer in the loop | both parse |
| 2 · Resolve references | code | loop checks IDs it allocates | 0 real unresolved refs; 35 built-in trigger refs the guide's reader wrongly flags |
| 3 · Canonicalize | code | stableJson + hash in shadow evidence | sha256:9b23… → sha256:d752… |
| 4 · Diff + blast radius | code | entityDiff in shadow evidence | 17 added, 105 changed (consent status on all 105, 41 renames, 11 folder moves) |
| 5 · Hard stops | code | not mapped shadowPolicy records none of these yet | consent touched · opaque code touched on 29 entities |
| 6 · Typed questions | Jev | frozen two atomic questions, rubric v2 | the six-question review set is a separate contract (field guide ch06), not run |
| 7 · Policy | code | keep / review / revert, shadow only; the loop decides by score | under the container guide, a hard stop would cap it at review |
| Audit | code | not run a separate workflow (ch08) | high 43→157: AUD-CON-02 0→114 (consent NEEDED with no types), unused triggers 4→6 |
To raise the consent dimension, the loop set consentStatus: NEEDED and stopped there. The scorer counts that as 100%. The guide's audit counts a NEEDED tag with no consent types as high severity. BLADE went from 105 medium findings (no consent setting) to 114 high ones; HRE from 0 to 20, and HRE's winner also added 2 Custom HTML or Custom JavaScript entities, which the guide treats as a hard stop. AutoLoop turns this into a how-to-work rule below. The scorer fix goes to the owner, because AutoLoop may not touch evals/.
AutoLoop may change one section of each program.md: "Edit Strategy", the how-to-work part. Constraints, the mutation budget, stop conditions, the weights and evals/ are fixed rules a human owns. The autoloop-scope check applies each proposal to a copy and confirms every other section is byte-identical. Nothing has been written to program.md.
parseProgram() in run-gtm-loop.ts reads only the template and snapshot paths. The strategy order and constraints are hardcoded arrays, and pickStrategy() ignores the strategy order anyway. Until the loop reads "Edit Strategy" from program.md, an AutoLoop rewrite, or a human one, changes nothing. That is the first fix in the plan. The second is recording what each reverted round tried, because AutoLoop can only see 2 of its 3 habits without it.
Box exists and runs. arb CLI with 32 checks tied to the field guide's chapters, the Jev guide's reader and audit vendored in box/jev/, AutoLoop analyzer, Worker with SSE stream, D1 schema, box.yml CI, and a recorded proof run on the real repo. Branch feat/autoresearch-in-a-box.
Make the code match the guide. Add .claude/settings.json denying edits to evals/**, the Jev rubric and box/. Rewrite parseProgram() to read Constraints, Mutation Budget and Edit Strategy, and make pickStrategy() follow them. Write per-dimension floors. Refresh both snapshots (the loop exits on them today). Fix HRE's weights table or add Google Ads data.
Record every attempt. Each round writes the field guide's receipt: strategy, operations, hashes, validator result, candidate dimensions, QA status and shadow route, reverted or not. Make a revert count as a regression, as chapter 04 says. Keep manifest.json pointed at the best winner. Correct the chapter 04 first-run row.
Proposals as pull requests. arb autoloop --pr opens a PR touching only Edit Strategy, with the evidence rounds in the body. Only the owner merges. Rerun BLADE with the merged habits and compare against 77.3%.
Climb the ladder one rung at a time. Stage 1: fix the shared reader so built-in triggers resolve, and test the evidence builder. Stage 2: map the container hard stops into shadowPolicy() as a versioned shadow change, and record the audit delta in each receipt. Then preview-mode QA per kept round and jev-site-scan. The six-question quiz is a separate contract to freeze and evaluate. Enforcement waits for stage 3: reviewed real labels, Preview evidence and the untouched holdout.
One link per client. Deploy the Worker, point CI at it, and publish the guide under guide.organizedai.vip/gtm-autoresearch/box. Package box/ so a new client folder gets the same gates.
Paste into Claude Code on the Mac. The env vars for Claude Code Web are at the end of the block.
cd /Users/supabowl && claude --dangerously-skip-permissions Work in /Users/supabowl/gtm-autoresearch (clone github.com/Organized-AI/gtm-autoresearch if missing). Check out branch feat/autoresearch-in-a-box. First step: apply the Organized Codebase agent templates to .claude/ before writing code. Then read CLAUDE.md, AGENT-HANDOFF/CURRENT-STATE.md, and the field guide chapters at guide.organizedai.vip/gtm-autoresearch (01-08). Where the guide and code disagree, fix the code unless the guide's number is the error. Context: box/ is "Autoresearch in a Box", an FDE package that maps the Karpathy loop + AutoLoop onto this repo. The inner loop (scripts/run-gtm-loop.ts) is unchanged. box/ adds `arb check` (32 checks, gates G0-G4, each tied to a field guide chapter; G4 runs the Jev container guide's reader and audit from box/jev/), `arb autoloop` (reads loop-results ledgers, finds recurring habits, proposes a rewrite of the "Edit Strategy" section of program.md only), a Worker with D1 + SSE run view, and .github/workflows/box.yml. The recorded proof is box/proof/2026-10-03-gtm-autoresearch.jsonl: 13 pass, 11 fail, 3 manual, 5 not built. Deploy first: 1. cd box && npx wrangler d1 create autoresearch-in-a-box, put the id in wrangler.jsonc, npx wrangler d1 migrations apply autoresearch-in-a-box --remote 2. npx wrangler secret put ARB_INGEST_TOKEN (openssl rand -hex 24), node site/build.mjs && npx wrangler deploy 3. From the repo root: ARB_URL=<worker url> ARB_INGEST_TOKEN=... node box/cli/arb.mjs check all, then arb autoloop. Confirm /run/ streams both live. 4. cp box/templates/box.yml .github/workflows/box.yml (the GitHub App here could not push workflows), then gh variable set ARB_URL and gh secret set ARB_INGEST_TOKEN on the repo so box.yml streams. Then work gate by gate. A gate is done only when `node box/cli/arb.mjs check <gate>` passes. Never edit box/checks to make a check pass. G1 Copy box/templates/claude-settings.json to .claude/settings.json (deny edits to evals/**, box/checks/**, box/jev/**, the Jev rubrics). Rewrite parseProgram() so Constraints, Mutation Budget, Stop Conditions and Edit Strategy come from program.md, and make pickStrategy() honor Edit Strategy (guide ch01). Add per-dimension floors to program.md and reject candidates that cross one (guide ch02). Keep tests green and add one test proving an Edit Strategy change alters the chosen target. Refresh both ads snapshots (both are partial and months old; the loop exits). Fix HRE: its table lists the 11 dimensions it is scored on, or the snapshot gains Google Ads data. G2 Every RoundResult carries the guide's receipt (ch04): strategy, operations, baseline/candidate hashes, validator result, candidateScore, candidateDimensions, failingDimensions, QA status, shadow route; reverted rounds too. Make a revert increment regressionCount so MAX_REGRESSIONS = 3 consecutive reverts works as the guide says. Update manifest.json "best" automatically (BLADE is stale: 76.8 vs 77.3). Correct guide ch04's first-run row to the ledger: 3 kept, 0 reverted, 2 provider failures. G3 Add `arb autoloop --pr`: writes the proposal to a branch and opens a PR that changes only the Edit Strategy section, with the evidence rounds in the body. Only the owner merges. Then rerun BLADE with the merged habits and report the score against 77.3%. G4 Judgment, shadow first (guide ch05-08 + guide.organizedai.vip/jev-container-json). Stage 1: fix the shared reader so GTM built-in triggers (2147479553, 2147479572, 2147479573) resolve, then copy it into box/jev/. Stage 2: map the container hard stops (consent touched, opaque code touched, unresolved refs, partial export) into shadowPolicy() as a versioned shadow change with reason codes; record the audit delta (today BLADE high 43→157) in each receipt. Do not enforce either. Per kept round, run a Playwright check in GTM preview mode and pass it as qa; call jev-site-scan for configured vs observed. Keep the frozen two-question contract; treat the six-question set as a separate contract to freeze and evaluate. Never open the holdout. Enforcement waits for stage 3 (reviewed real labels, Preview evidence, written thresholds) and a separate acceptance workflow. Rules: no time estimates anywhere. Never publish a GTM container. Commit after each gate passes and update AGENT-HANDOFF/CURRENT-STATE.md. Push to github.com/Organized-AI/gtm-autoresearch and open a PR against main. # Claude Code Web env vars CLOUDFLARE_ACCOUNT_ID=<your account id> CLOUDFLARE_API_TOKEN=<Workers Scripts:Edit, D1:Edit, Account Settings:Read> ARB_INGEST_TOKEN=<same value as the Worker secret> ARB_URL=https://autoresearch-in-a-box.<subdomain>.workers.dev ARB_REPO=. GITHUB_TOKEN=<repo + workflow scope on Organized-AI> MUTATION_PROVIDER=claude MAX_ROUNDS=30