Shared-Memory Parallelism: Why the Simulation Moved to Rust

By Robin Dowling · 3 months ago

The whole point of moving Wilden's backend from Node to Rust was a payoff I couldn't get on the old runtime: real shared-memory parallelism over a single live world, distributing state workloads across every core of the machine, without changing a single byte the browser client sees.

Why Node couldn't do this

The simulation is a few thousand organisms that live entirely in memory. Every tick, each one reads its neighbours' state (energy, position, mass, fear, what they're carrying), decides what to do, and acts. Keeping that state resident and hot is the whole game: it's what keeps per-tick latency low enough to broadcast state updates at 10 fps.

Node has one JavaScript thread. You can add worker_threads, but they don't share mutable memory. Each worker has its own heap, and to move data between them you either copy it (serialize, message-pass, deserialize) or confine yourself to a SharedArrayBuffer of raw bytes with no structured objects on top. For a workload whose hot path is "read arbitrary neighbours' rich state, mutate the world," both options are poison: serialization on every tick would dwarf the computation it's trying to parallelise, and a flat byte buffer throws away every type the simulation is built on. So in practice the tick stayed single-threaded, and "add more clients" literally meant "steal time from the simulation."

Rust removes the constraint. Threads share one address space; the borrow checker and the Send/Sync traits prove at compile time that concurrent access is safe, so I can hand many cores read-only references to the same live world and let them work in parallel with zero copying. That's the capability the migration was really for.

The shape of a tick

A tick has two halves:

1. Decide (CPU-bound, and independent per organism so it splits cleanly across cores): every organism reads the world and produces a list of intents, such as "consume this soil," "fight that rival," "reproduce with this mate," "move here." No organism mutates anything but itself during decide.
2. Commit (sequential, cheap): apply all the intents in a single deterministic order, resolving conflicts (two organisms eating the same soil, a kill that invalidates a later intent) by fixed rules.

Decide is where the cores go. Commit stays on one thread on purpose: its order is part of the contract.

Making the tick safe to parallelise

You can't just throw par_iter at a loop that reads and writes shared state mid-walk; you'd get a different result every run. Three changes made the decide phase order-independent by construction, so that parallel execution is provably identical to sequential:

• Per-entity RNG streams. The old design drew from one global random generator in walk order, so any reordering changed every subsequent draw. Now each organism carries its own stream, seeded from splitmix(world_seed, num_id). A draw's result no longer depends on when in the tick it happens. (~32 bytes per organism, about 1.6 MB at 50k, cheap.)
• Frozen tick-start reads. During decide, every cross-entity read sees the neighbour's value as it was at the start of the tick, not its half-mutated mid-tick value. Each organism still reads and writes its own live state (it exclusively owns itself), but it can never observe another organism's same-tick changes. This is built as a compact, read-only snapshot of the fields the decide phase consumes.
• num_id-ordered commit. Intents are applied in a canonical order keyed by a monotonic birth sequence number, regardless of which thread produced them. Conflicts resolve the same way every time: soil over-draw clamps in actor order, kills are first-actor-wins, an already-pregnant mate drops later reproduction intents.

With those in place, a decide worker reads only frozen neighbour data plus its own slot, and emits intents into a per-worker buffer. Nothing it touches is contended. The proof I leaned on: run the decide walk backwards and the committed world is bit-for-bit identical. If order doesn't matter, threads don't either.

Distributing the work across cores

Once correctness no longer depends on how the work is partitioned, partitioning becomes a pure load-balancing and cache-locality choice, and the simplest option wins.

The decide phase is a rayon par_iter_mut over the organism store, sharded by store index (organisms.len() / k chunks), not by map position. Each worker takes a disjoint set of &mut Organism slots and shares one read-only neighbour view. Store-index sharding gives near-perfect load balance no matter how organisms cluster on the map, with zero rebalancing machinery. Intents fold into a per-worker buffer, then reduce into one before the sequential commit re-sorts them into canonical order, so the result is independent of how the work happened to shard across threads.

I deliberately rejected the fancier option. Spatial or checkerboard scheduling (coloring map cells so distant ones run concurrently) only pays off on much larger maps, couples correctness to geometry, and would still be scheduling-order-sensitive for RNG. The decide/commit model needs none of it. If very large worlds ever demand cache locality, spatial sharding can be reintroduced as an optimisation on top of the same correctness model.

Three pools, one core budget

A server core is wanted by three different consumers, and the trap is letting each grab "all cores":

These are sized as one partition, not three independent pools, because compute and transport contend in the same window: the transport plane builds tick N-1's frames while the engine computes tick N's decide. So a boot-time core budget resolves all three from a single number and logs what it picked. reserved_cores (default 2) is headroom held out of the compute pool for the engine thread and the transport plane's CPU burst, deliberately not one reserved core per tokio worker, because transport mostly yields and mild oversubscription is optimal there.

A second decoupling makes the client count irrelevant to tick latency: at the end of each tick the engine publishes an immutable Arc<SimSnapshot>, and every broadcast reads only that snapshot, never the live sim, never a lock the engine holds. Adding clients can no longer steal simulation time, which was the original sin of the Node design.

Performance

Two things are worth measuring separately: how expensive the tick itself is (compute), and how many clients a box can serve (transport).

Compute cost. A single-threaded tick is super-linear in organism count, because the dominant cost is the per-organism neighbour scan, which grows with neighbourhood density:

At 50k a single-threaded tick is ~40× over the 100 ms frame budget. That gap is the entire reason the decide phase runs in parallel: spreading it across the compute pool is the lever for larger worlds. (These are single-thread reference numbers; the multi-core decide is still being tuned, so I don't yet have clean cross-core tick benchmarks to quote.)

Transport scaling. Here I do have measured cross-core data, because the snapshot decoupling means client count no longer steals tick time, so the only question is how many concurrent SCOPED clients a box holds at the 10 fps cadence. Measured over the full path (TLS + engine) at thewilden.com, with the load generator running off-box:

The 2-vCPU box (which collapses to a sequential decide) holds full cadence only to ~200 clients. The 4-vCPU box serves 1,000 concurrent clients at 9.3 frames/s and degrades gracefully past that rather than collapsing, because a bounded drop-oldest outbound queue means a slow client misses frames instead of blocking the engine. The jump is roughly 4× the throughput at a given load, because the parallel planes gain workers, not just clock.

Matching the machine

The deployment target is the GCP C4A (Axion, Arm Neoverse V2) performance series, c4a-standard-4 to -16 depending on world size. The reasoning is specific to this workload:

• Dedicated physical cores, no SMT. On C4A every vCPU is a full physical core, so the engine, compute, and transport pools run on real hardware rather than sharing an execution unit via hyper-threading. That kills the per-tick jitter that SMT siblings cause on standard x86 VMs, and for a server that holds a tick rate for months, consistency beats peak clock.
• Performance tier, not efficiency. C4A and N4A are both Axion; the difference is sustained throughput and memory bandwidth, which is exactly what a memory-bandwidth-bound neighbour scan cares about.
• Titanium network offload. NIC packet processing is offloaded off the host CPU. Note, though, that TLS termination still runs on the guest, and the 1-core load test found CPU-side TLS handshakes to be the connection ceiling, which is precisely why more performance cores help under a connection burst.

The compute pool auto-sizes to available - reserved_cores, so scaling the world up is mostly a matter of picking a bigger machine: 4, 8, then 16 vCPU gives a 2-, 6-, then 14-worker decide pool, no config change.

Deploying it

The binary detects its allowance, not the host. std::thread::available_parallelism() honours cgroup CPU quotas and affinity masks on Linux, so a --cpus-limited container reports what it's actually been granted, and every pool sizes from that. The same image adapts to a 1-core box (where it runs a bit-identical sequential decide) or a 16-core box with no change, and parallelism.enabled: false even forces the sequential path explicitly. Deployment is a Docker image rolled out via Terraform to a long-lived Compute Engine instance, where an always-on server earns a committed-use discount that makes the performance tier affordable.

The payoff

Same seed, same world, on any core count: that's the determinism contract the parallel engine holds, verified by a golden-master digest and a statistical-faithfulness harness. The browser client never noticed; the wire format is byte-compatible throughout. What changed is that the simulation now uses the whole machine, the way it always wanted to, and the only reason it can is that Rust lets many threads share one live world safely. That capability was the entire argument for the migration, and this is it, working.