Building a C Compiler with a Team of Parallel Claudes
Anthropic researcher Nicholas Carlini tasks 16 parallel Claude agents with writing a Rust-based C compiler from scratch, and documents what it actually takes to run autonomous agents without human oversight.
Running an agent in a loop and calling that “autonomous development” is not new — the model runs for a bit, hits a wall, and waits for a human to unblock it. What separates an agent team from that pattern isn’t the loop or the parallelism. It’s whether the feedback environment is strong enough that each agent can tell, without asking anyone, whether it’s actually making progress.
Source
Building a C compiler with a team of parallel Claudes
By: Anthropic
Published: Feb 05 2026
Type: Engineering article
Saved because the lessons about designing test harnesses for autonomous agents — not for humans — apply to any long-running agentic system, well beyond the compiler benchmark that frames the piece.
What it takes to run 2,000 agent sessions without a human in the loop
Nicholas Carlini’s piece is ostensibly about a C compiler. It’s really about what you have to build around an agent to make 2,000 consecutive Claude Code sessions produce coherent output instead of diverging chaos — and where that approach hits a ceiling even today.
The basic scaffolding is a bash loop. Run Claude, when it exits, run it again. Claude is prompted to break the problem into small pieces, track its own progress, and keep going until the work is done. The loop never terminates — Claude has no choice but to keep working. (In one instance, Claude accidentally ran pkill -9 bash, killing its own process and ending the loop. Not a planned exit.) That’s the easy part. The hard part is everything that makes the loop useful rather than circular.
Parallelism uses a bare git repository as the coordination layer. Each of the 16 agents gets a Docker container with the repo cloned locally. To claim a task, an agent writes a text file to current_tasks/ — a convention that doubles as a locking mechanism, because git’s atomic push semantics force the second writer to the same task to back off and pick a different one. Agents work, merge in changes from other agents, push, and remove the lock. Merge conflicts are frequent; Claude handles them. There is no orchestration agent. Each instance independently decides what the most obvious next problem to work on is.
The first and most important lesson is about test quality, and it’s more severe than it sounds: Claude will autonomously solve whatever problem the test harness specifies. If the harness verifies the wrong thing, you get 100,000 lines of code that passes tests by solving the wrong problem. Carlini found himself continuously improving the test suite as he identified failure modes. Near the end of the project, Claude started breaking existing functionality each time it implemented something new, which required building a proper CI pipeline to prevent regressions. The test harness is not scaffolding around the real work — it is the real work.
The second lesson is about designing feedback for the actual consumer of that feedback. When you write a test harness for yourself, you bring context — you can read a stack trace and infer the failure from memory. Claude gets dropped into a fresh container with no prior context on every session. The harness needs to orient it from scratch. That means up-to-date README and progress files updated continuously, and test output designed to be grep-able: if there’s an error, the word ERROR appears on the same line as the reason. Pre-computed aggregate statistics mean Claude doesn’t waste context window space recomputing things it could just read.
Context window management explains another design choice: the harness uses a --fast flag that runs a 1% or 10% random sample of tests by default. Without it, Claude would spend significant runtime running the full test suite instead of fixing things. The sample is deterministic per agent but randomized across VMs, so the full suite gets coverage across the fleet without any one agent burning its context on it.
The parallelism ceiling appeared at 99% test pass rate. With hundreds of independently failing tests, each agent naturally picks a different one — parallelism is trivial. Compiling the Linux kernel is a different shape of problem: one giant task where every agent hit the same bug, fixed it, and overwrote each other’s changes. Sixteen agents running didn’t help because each was stuck on the same bottleneck. The fix was using GCC as an oracle: most of the kernel compiled with GCC, and only a subset of files with Claude’s compiler. If the kernel built under that split, the problem wasn’t in Claude’s files. Each agent could then fix different bugs across different file subsets in parallel, with delta debugging afterward to find pairs of files that failed together but succeeded independently. The technique generalizes: at scale, parallelism requires task decomposition that makes each unit of work genuinely independent. When that decomposition doesn’t exist naturally, you have to engineer it.
Specialization is the other dimension of parallelism. One agent was tasked with finding and coalescing duplicate code. Another with improving compiler performance. Another with improving code quality from a Rust developer’s perspective. Another with documentation. Multiple concurrent agents doing different things on the same codebase is not just faster — it addresses the tendency for LLM-written code to re-implement existing functionality when not actively monitored.
The compiler itself is the benchmark: 100,000 lines of Rust, clean-room implementation with no internet access during development, capable of building Linux 6.9 on x86, ARM, and RISC-V. QEMU, FFmpeg, SQLite, Postgres, Redis. 99% pass rate on the GCC torture test suite. It runs Doom. The remaining limits — no 16-bit x86 real mode code generation (calls GCC for that phase), no own assembler or linker, generated code less efficient than GCC with all optimizations disabled — are genuine model capability limits at this point in time. Carlini tried repeatedly to fix several of them and couldn’t. These are not engineering oversights.
Total cost: 2 billion input tokens, 140 million output tokens, just under $20,000 over two weeks. The conclusion is uncomfortable and honest: this was barely possible in early 2026, it surprised even the researcher who built it, and the rapid pace of both model capability and scaffold design means we’re entering a period where a large amount of software will be written without a human ever reviewing it.
My take
The key asymmetry this reveals: agent capability is now ahead of our ability to specify what we actually want. The bottleneck isn’t the model — it’s the test harness. Writing tests hard enough to prevent the wrong solution and clear enough to orient an agent with no context is harder than writing the code itself. That inversion is new, and most workflows haven’t caught up to it.
Where this is useful
- Designing harnesses for any long-running agentic workflow where continuous human oversight isn’t the plan
- Thinking through task decomposition strategies that actually enable parallel agent work rather than just running duplicated effort in parallel
- Getting a grounded sense of where current model capability limits actually are versus where the headlines put them
- When NOT applicable: short-horizon tasks where a single agent session plus a human reviewer is both cheaper and more reliable — autonomous orchestration overhead only pays off when the task duration genuinely exceeds human attention span
Apply this: Before running an agent on a multi-session task, ask: “If the agent solves the literal problem this test verifies, does that actually solve my problem?” Fix the test before running the agent.
What I want to remember
Remember: Claude solves exactly the problem the test harness specifies — not the underlying problem you have in mind — so test quality directly sets the ceiling for what autonomous agent output is worth.