Python 3.13 Gets a JIT
A CPython core developer walks through the copy-and-patch JIT added to Python 3.13, explaining what stencils and holes are at the machine-code level and why a 2-9% initial gain is the right starting point.
CPython has run bytecodes through a C dispatch loop since the 1990s. Every attempt to add a JIT either bloated startup time, ballooned memory, or demanded shipping an external compiler alongside the runtime. The change that landed on Christmas Day 2023 does none of those things — and that restraint is exactly what makes it worth understanding.
Source
By: Anthony Shaw (CPython core developer)
Published: Jan 09 2024
Type: Engineering article
I saved this because it is the clearest public explanation of copy-and-patch JIT design — a technique only two years old when this was written — and it grounds the explanation in CPython’s specific implementation rather than staying at theory level. It also corrects the PyPy comparison before anyone has a chance to make it.
How copy-and-patch JIT works
CPython has always interpreted bytecodes through a dispatch loop compiled in C. When your code calls a function, Python doesn’t run machine code written for that function — it runs a generic C loop that reads each bytecode, matches it against an instruction handler, and dispatches to that handler. The loop overhead is trivial for one call. For ten million calls in a tight benchmark loop it accumulates, and more importantly the loop prevents the optimizer from knowing anything specific about the function’s type behavior or value flow.
The obvious answer — compile bytecodes to machine code — has a well-known price list. A full JIT needs an intermediate language, a code generator per CPU architecture, and memory proportional to what it has compiled. Research cited in the article shows GraalPy and Jython can start 100x slower than CPython and consume an extra gigabyte of RAM to compile user code. For an interpreter shipped to hundreds of millions of devices from Raspberry Pi to M-series Macs, those costs are not payable.
Copy-and-patch solves this by doing most of the work at CPython’s own build time, not at user runtime. When CPython is compiled from source on the CI machines that produce python.org releases, each bytecode handler is compiled by LLVM into a “stencil” — a fixed sequence of machine-code bytes with certain values left as zeros. Those zeros are called holes. For LOAD_CONST, the holes are the operand index and the address of the next instruction. Every stencil for every bytecode is stored as a byte array in a generated file called jit_stencil.h, which ships inside CPython.
At user runtime, when a function becomes hot enough to JIT, CPython allocates executable memory and walks the function’s bytecode list. For each bytecode, it copies the pre-built stencil bytes into that memory and patches the holes — writing the actual operand value and the actual next-instruction address for this specific function. What results is a flat, sequential block of machine code, inlined in order, with the dispatch loop gone. The second time you call the function, CPython jumps directly to that block.
Shaw builds this intuition first with a Python-level analogy: a function that walks a function’s bytecodes and emits a sequential Python code string with values inlined rather than dispatched. It is not the same thing as x86 stencils, but the structural point holds — “copy a template and substitute the operands” produces sequential code, while “loop over instructions and dispatch” doesn’t. He is upfront that the analogy is simplified, which makes it more useful rather than less.
The tradeoffs are explicit and worth naming. Copy-and-patch does not perform classical JIT optimizations: no constant propagation, no type specialization across call sites, no inlining across bytecode boundaries. Every stencil is a verbatim copy of its C handler with operands substituted. This is why the initial benchmarks show 2–9% improvement rather than 2–9x. CPython is already a C program compiled to native machine code. The 2–9% is improvement on top of that baseline, at zero startup cost for end users and with no external dependency at runtime. The LLVM toolchain is only needed when building CPython itself — the person running Python never touches it.
Shaw is direct about what those numbers mean: the initial gains are not the point. All of the large optimizations that will eventually matter — type-specialized stencil variants, reference-count elimination on provably-constant paths, cross-stencil dead code removal — are not possible inside the interpreter loop. They require controlling the emitted machine code per function. The copy-and-patch JIT creates that layer. In 3.13 the layer is essentially empty by design; subsequent releases are where optimizations get stacked. The JIT is also opt-in in 3.13 (--enable-experimental-jit) and only activates for code containing JUMP_BACKWARD, which means loops — the hottest paths.
The misunderstanding worth correcting: comparing CPython’s 2–9% to PyPy’s 5–10x speedups frames the wrong question entirely. PyPy is a separate implementation with fifteen years of tracing JIT work, designed from the ground up to compile Python semantics. The correct comparison is CPython with JIT enabled versus CPython without JIT enabled on the same build. On that comparison the gains are real, the costs are minimal, and the mechanism makes structurally available a class of optimization that was not available before.
My take
What makes this article worth rereading isn’t the technical specifics of stencils — it’s the framing discipline. Shaw spends as much time explaining what copy-and-patch cannot do and why those limitations are acceptable as he spends explaining what it can do. That is rare in engineering writing about new features, and the result is an explanation that held up: everything the article promises has happened, and nothing it was careful not to promise has been quietly assumed.
Where this is useful
- Tracking the CPython 3.13 → 3.14 → 3.15 performance arc — understanding what the JIT layer is before later optimizations land on top of it
- Making sense of why “Python gets a JIT” headlines don’t automatically mean “Python is now fast for compute-heavy work”
- Designing language runtimes where startup time is a hard constraint: copy-and-patch gives a concrete answer to “how do you JIT without shipping a compiler at user runtime?”
- Less applicable if you need large Python speedups today — the 2-9% initial gain doesn’t change the practical ceiling for computation-heavy workloads
Apply this: Before benchmarking
--enable-experimental-jiton 3.13, confirm the hottest code path containsJUMP_BACKWARD— the 3.13 JIT only activates for loops. A bottleneck in a single linear function won’t be touched yet.
What I want to remember
Remember: A JIT that produces 2-9% gains at launch is not a weak JIT — it is a first layer. The meaningful test of a new JIT is not the size of the initial improvement but whether it creates a surface where larger optimizations can live that were structurally impossible before.