Migrating the Backend to Rust, One Module at a Time

By Robin Dowling · 3 months ago

The backend that runs Wilden started life as a Node/TypeScript server: it owns the world, ticks the simulation, and broadcasts state to every connected client. It worked, but a simulation of thousands of organisms all obeying the same rules is exactly the kind of workload that wants real concurrency and predictable performance. So I decided to migrate the entire backend to Rust without ever changing how the simulation behaves, and without touching the browser client.

It took hundreds of pull requests. Here's the course I followed.

1. Refine and simplify first

You don't port a mess; you port a contract. Before any Rust was written, I spent time simplifying the existing code: collapsing redundant entity-management paths, clarifying module boundaries, deleting dead branches. A migration is the worst possible time to discover that a module's behaviour was never really pinned down. The cleaner the TypeScript, the smaller the surface I'd have to reproduce exactly in Rust.

2. Get close to 100% meaningful test coverage

This was the precondition for everything else. I built out the test suite until nearly every observable behaviour was covered. Not line count, but contract: normal paths, boundary conditions, and the corner cases callers actually depend on. The client got its own suite of 300+ tests; the backend's behaviour was locked down module by module.

The reason coverage has to come first is subtle. If you write the tests after the Rust exists, they only prove the Rust does what you thought you wrote. Tests written against the original TypeScript, before any Rust is in sight, describe the real contract, untainted by the new implementation. They become the reference the Rust has to satisfy.

3. Migrate one unit at a time, behind a bridge

I used napi-rs as a foreign-function bridge so Rust could be called directly from Node. Then I ported the backend one module at a time, in dependency order, leaves first.

The utilities went early: hashing, quantization, comparison, the RNG, compression, time, IDs, binary serialization, Perlin noise. Then the domain core: energy and physics, environment elements, the organism (reproduction, movement, interaction, awareness), the world, statistics. Then the server layer: the client tracker, lineage, config, broadcast, state, routes.

Each module followed the same recipe. Port it to Rust, expose it through napi, and, critically, turn the original TypeScript file into a pure re-export of the Rust binding. No logic left in TypeScript; the .ts file becomes a one-line forward to the Rust implementation. Callers don't know or care that the work now happens in Rust.

One rule made this safe to do incrementally: the old implementation never came out until the new one was independently proven, with old and new running side by side behind the bridge for the entire migration. None of this touched production. The whole effort lived on a long-lived feature branch, and the live game kept running the Node backend until the very end.

4. Node tests cover both implementations

This is the trick that made each step verifiable. Because the TypeScript module became a thin re-export of the Rust binding, the same TypeScript tests now ran against the Rust code, through the napi seam. I didn't rewrite them. I re-pointed them.

The moment a Rust port diverged from the original TypeScript behaviour (a rounding difference, an off-by-one in an iteration order, a different RNG sequence) the existing test went red. The contract I'd captured in step 2 was now actively policing the port, automatically, on every commit. No "does this look right?" judgement calls.

5. Duplicate the tests in Rust, then add more

The napi-seam tests prove parity, but they test Rust as if it were TypeScript, through a foreign-function boundary. So alongside them I wrote Rust-native tests directly against the kernel (idiomatic assertions, real Rust types, no FFI overhead), duplicating what the jest suite covered and then going further into the corners that are awkward to reach from JavaScript. cargo clippy -D warnings became a non-negotiable fifth check next to type-check, the two test suites, and the build.

The two test layers run together, not instead of each other. A dedicated parity sweep made sure every assertion the jest suite covered had a Rust-native equivalent before the jest coverage could ever disappear. The TypeScript tests stay until the whole migration is done; pulling one early would remove a cross-validation signal that catches regressions introduced by later work.

6. Faithful first, idiomatic second

The first Rust draft of any module was a deliberately literal translation of the TypeScript: same control flow, same field names, same order of operations. Faithful, even where it was ugly. That's what kept the parity tests green and made each diff reviewable.

Then, with the behaviour pinned, I optimized, in a fixed priority order: raw performance, then idiomatic Rust, then developer ergonomics. Sometimes being idiomatic meant breaking from the Node architecture entirely: a pattern that made sense under a single-threaded garbage-collected runtime is often the wrong shape in Rust. Where the faithful translation fought the language, I changed the structure to fit Rust, and recorded why in an architecture decision record, so the deviation was a conscious, documented choice rather than drift.

7. Hold byte parity, and record every deviation

The governing invariant was byte parity: the same seed must always produce the same world, byte for byte, before and after a change. A determinism digest over the whole world state was the canary throughout. If a full simulation run was identical before and after a refactor, behaviour was preserved no matter how deep the change went.

Byte parity isn't free, though, and a few changes were worth deviating for. Each one was a sanctioned break: written up in an ADR with its rationale, gated by a statistical-faithfulness harness (proving the simulation's character, such as population curves, births, deaths by cause, and species counts, was unchanged even when the exact bytes moved), and re-baselined in the same PR. The rule was simple: you may leave byte parity, but only deliberately, and only on the record.

8. Remove the bridges, one by one

With everything ported and double-covered, the next phase was tearing out the bridge. This was the hard, behaviour-changing part, so it moved slowly and deliberately: one module's seam at a time, each with its corresponding jest tests retired only once the Rust-native coverage stood on its own.

I split each module into a pure-Rust kernel and a thin napi mirror, then collapsed the side effects (logging, stats, state mutation) down into the kernel so the TypeScript shims had nothing left to do. The biggest piece was dismantling a dependency-injection "bag" that had been threaded through every cross-organism interaction: each interaction was rewritten to operate directly on kernel-native state, removing the foreign-function calls from the simulation's hot loop entirely.

Finally, the cutover: a pure-Rust engine running on an axum/tokio HTTP and WebSocket server, the production entry point flipped to a standalone evolution-server binary, and then the napi seam and the entire TypeScript/jest layer deleted. The mapping layer was gone.

9. Update the docs, then review the whole thing again

A migration this long leaves a trail of prose that quietly goes stale: architecture docs, developer guides, config references, code comments that still describe a Node server. So a documentation pass was part of the definition of done, with every artifact rewritten to describe the system as it is now, in the present tense, and the why living in the ADRs.

Then a global review of the entire migration, not module by module this time, but the whole Rust codebase as one artifact. With the bridges gone and the architecture free of its Node-shaped constraints, I optimized again, in the same priority order as step 6: performance, idiomatic Rust, developer ergonomics. Things that had to stay literal while the seam existed could finally take their natural Rust shape.

Review and fix until it converges

"Tests pass" was never the bar for merging. Every change set went through repeated review-and-fix passes, judging migration fidelity, idiomatic style, performance, and corner-case coverage, and I kept looping until a fresh review pass found nothing. Green tests prove the code does what the tests check; a clean review proves it's actually right. Only after the loop converged did a change set get manually tested in the running client, and only then merged, before moving on to the next task.

How it was actually run

I didn't hold this whole thing in my head. The migration was driven by a single plan document that doubled as a state store (every module's status, every decision, every deferral) and an execution engine (a fixed per-module recipe and a "find the next task" router). Each work session, most of them run by an AI agent, started cold, read the plan, picked the next unstarted module, did exactly one PR's worth of work, and stopped at a human merge gate. The plan and the merged PRs were the handoff between sessions: no meeting, no lost context, and several lanes safely in flight at once across the team.

That mattered more than it sounds, because of how I worked on it: offline Claude sessions from my phone, checking in when I had a spare moment, nudging a plan or approving a step in a queue, then testing the night's PRs after the kids were asleep. A migration that lives entirely in the repo (plan plus merged PRs) is one you can drive in fifteen-minute slices without ever rebuilding context. That's a whole post of its own, but it's the reason a migration this size stayed coherent.

Results

• The backend is 100% Rust: a single evolution-server binary serving HTTP and WebSocket over axum/tokio.
• The only JavaScript left is the browser client, which was never touched.
• Behaviour is preserved end to end: same seed, same world, before and after, with every deliberate deviation recorded in an ADR.
• Every step was verified against the original contract, reviewed to convergence, and signed off by a human before merging into the long-lived Rust branch. Production was cut over to Rust exactly once, at the end, in a single restart.

The simulation that's running right now, as you read this, is the Rust one. You can't tell, and that was the entire point.