Wilden

Seeing what an animal knows

By Robin Dowling · 4 days ago

Every animal in the world carries a small memory. It remembers where it found food, where its herd was, and which other organisms turned out to be good or bad to eat. Animals even share what they know when they meet. But until now, all of that lived on the server. If you controlled an animal, you had no idea what it knew. Now you can see it.

An animal

A memory of 25 things

An animal can hold at most 25 pieces of knowledge at once. Each piece is either GOOD or BAD, and belongs to a category: a position where it found energy, a position where it met its own species, or a kind of organism it has learned to eat or avoid. When the memory is full, something old is forgotten to make room for something new.

This knowledge already shapes behavior. A hungry animal heads back toward a place it remembers being fed. A lost one drifts toward where it last saw its kind. The player just couldn't see why.

Most of it is hearsay

Then I measured what the animals actually know. About 80% of the herd locations an animal remembers are places it has never been. They're hearsay, passed from animal to animal, some of it passing generations. Nothing marks a memory as secondhand, so the animal follows all of it with the same confidence.

That's by design. Most of what any of us knows works the same way.

Who does the work?

The server supports up to a thousand players controlling animals at the same time. So the first question was where to decide which arrow points where.

The arrows depend on the animal's position, and the animal moves every tick. If the server picked the nearest remembered place for every player on every tick, it would spend most of its time redoing the same sums. The client, on the other hand, already knows exactly where its animal is, sixty times a second, and only ever serves one player.

So the server only sends what the animal knows, and the client does the rest. The memory is tiny, so sending all of it is cheap. And it only changes every few seconds, so the server only has to send it when it actually changes.

Sending only what changed

Each animal already keeps a running checksum of its memory, a single number that changes whenever anything is added or forgotten. Every tick, the server compares that number with the one it saw last time. If they match, it's done. That one comparison is all a quiet tick costs.

When the checksum does change, the server filters the memory down to what the player will actually see: only the GOOD positions. If the filtered result is the same as last time, for example because the animal only learned about a bad place, nothing is sent. Only a real change goes out, stamped with a new version number, so the client always knows which copy is newest.

The server remembers just one thing per player: what it last sent for the animal they control. Switch animals and it's overwritten. Let go of control, or disconnect, and it's gone. It can never grow into a list.

The butterfly effect

By Robin Dowling · 18 days ago

A harmless-looking function was quietly rolling dice. It was only used for stats, but every time it was called it nudged the organism's random number stream. After 27,000 ticks, one extra organism survived. That tiny change was enough to unravel the whole world.

The hidden side effect

Each organism has an objective, basically what it wants to do next. Eat, reproduce, rest etc. On paper, it's a simple check: look at the organism's energy and state, then answer.

But buried inside it was a hidden randomness roll. The function called the reproduction check, and that check advanced the organism's random generator. The result was only used to decide whether to log something as REPRODUCE or REST. It seemed harmless, but it wasn't. Every call used up a random number, and those numbers shape the future.

The drift

This function was called by the stats system, which runs on an interval. Most of the time, that extra roll never mattered. But one skipped draw changes the sequence for everything that follows. The organism picks a different direction, takes a different action, or reproduces at a different moment.

For a long time, nothing obvious happened. The differences were small. A little path change here, a tiny outcome change there. The world looked stable.

Then, around tick 27,300, a single animal survived when it should have died. That one extra life was enough to change the local balance. Its neighbors got less food. One failed to reproduce. A gap stayed empty. Another animal went hungry and wandered elsewhere. The ripple kept spreading.

An animal

The collapse

Nothing dramatic happened at first. The simulation still looked natural. But the small differences accumulated. Population mixes shifted. Species that had been in balance started pulling apart. The system had been stable for a long time, but only because every random choice happened in the same order. Once that order drifted, the whole world began to move away from its old equilibrium.

The fix was simple: split the pure "can this organism reproduce?" question from the randomness that decides whether reproduction actually happens. The stats code should not touch the random stream at all.

The deeper lesson is simple: if a function is named like a question, it should not secretly change the answer.

True evolutionary equilibrium

By Robin Dowling · 21 days ago

Five weeks. A hundred and fifty generations. The world is still alive, and it is not just surviving. It is thriving.

I have watched enough almost-stable worlds collapse that I did not want to declare victory early. Now, at 150 generations, I am confident. This is a system that has found its rhythm.

What equilibrium actually looks like

The definition is simple to state and difficult to achieve: offspring produce offspring, and those offspring thrive. No overpopulation. No overconsumption. Both plant life and animal life in balance, within their own domains and in harmony with each other.

Plant populations swell and recede in response to soil nutrients and competition for sunlight. Animal populations track the plant cycles with the characteristic predator-prey lag. Neither side crashes. Neither side overwhelms the other. Species still go extinct and new ones still form, but the system absorbs those shocks and keeps going. Dynamic equilibrium. A living system that corrects itself.

Migration across the world

Both plant and animal populations have migrated across the entire map over five weeks. I did not program migration patterns. The rules of survival pushed them there.

Plants spread through reproduction into less competitive ground. Over dozens of generations, vegetation crept across terrain that was barren at genesis. Animals followed on their own schedule, drifting toward areas with more food, splitting off from overcrowded regions.

The two migrations are independent but intertwined. Plants do not follow animals and animals do not follow plants, but each enables the other. Animals cannot survive without vegetation, and vegetation thrives when animal deaths return nutrients to the soil. They spread together without coordinating, each responding to their own survival pressures.

Speciation through mutation

Over 150 generations, small genetic variations accumulated. Populations that were once a single species drifted apart as individuals in different regions adapted to different conditions. When enough genetic distance built up, the clustering algorithm recognized them as separate species.

Most new species do not last. They cannot find mates, run out of food, and go extinct within a few generations. This mirrors real evolution, where the vast majority of species that ever existed are now gone. But some mutations land in favorable territory, build numbers, and establish themselves as permanent parts of the ecosystem. Several species in the current world were not in the original seed. They exist because a mutation fifty or a hundred generations ago happened to create something viable.

The right seed

Out of millions of possible starting configurations, this was the first seed that produced sustained life. Like life on Earth, everything needed to sustain it happened to be in the right place at the right time. The right spatial arrangement, the right genetic compositions, the right soil nutrient distribution. Change any one variable and the whole system collapses.

What makes finding the right seed so difficult is that every parameter is coupled to every other. Reproduction rates, digestion efficiency, mutation ranges, knowledge accumulation and sharing, fear responses, species compatibility. All interact in ways that are impossible to reason about in isolation. You cannot tune one without affecting all the others. The only way to evaluate a configuration is to run it and watch.

This seed, paired with the current parameter set, happened to produce a configuration where all of those interdependencies reinforce each other instead of undermining each other. It was not designed. It was random, like every other seed. It just happened to land in the narrow window where the rules of physics and evolution produce life instead of collapse.

Five weeks in

The world has texture now. Dense forests of vegetation and open areas where animals roam. Different regions host different species. The ecosystem has geography, sculpted not by design but by 150 generations of survival pressure.

I still do not know how long it will last. But five weeks of sustained, dynamic equilibrium is proof that the rules are capable of producing something that looks and behaves like a real ecosystem. Not for a few thousand ticks, but across generations that span weeks of real time. One random seed out of millions, landing in the narrow window where possible became actual.

Offloading the heavy lifting to Grafana Cloud

By Robin Dowling · 22 days ago

The simulation's metrics started as terminal logging. In the early days, the only way to know if life was emerging - if organisms were actually surviving, reproducing, and forming species - was to watch numbers scroll past in stdout. Birth counts, death counts, population totals, energy budgets. It was primitive, but it was enough to detect whether the world was alive or collapsing.

As the simulation grew more complex, so did the need to understand it. Terminal output became a custom metrics aggregation engine: a stats tracker built on HashMaps behind a Mutex, updated from the simulation's main loop every tick. The server serialized the current values into a compact format and broadcast them over a dedicated WebSocket channel to every connected browser client. The home page ticker, the in-simulation stats overlays - everything consumed this same feed. It tracked ~30 stats and it worked well. I always knew what was happening in every world.

But the engine had outgrown the server. The simulation was doing double duty: running the world and aggregating, serializing, and distributing all the stats data to every connected client. That's a lot of compute for a machine whose primary job is simulating evolution. The metrics were solid; they just needed to run somewhere else.

So I kept the same data but moved the heavy lifting. The stats pipeline is now Prometheus metrics scraped by Grafana Cloud.

How it grew

The progression was natural. Terminal logging told me whether life existed. The custom engine told me what was happening in detail. But both ran inside the simulation process, competing for the same CPU cycles. Adding a new metric meant touching the tracker, the serializer, and the client deserializer. There was no retention beyond the current moment - spawn a new world and the data was gone. There was no way to query historical trends or build visualizations beyond what the client hard-coded. The system was good at what it did, but it was doing it in the wrong place.

Prometheus on the server

The replacement writes metrics directly from the simulation engine using the prometheus crate's GaugeVec and CounterVec types. Point-in-time state - alive organisms, species counts, energy levels - uses gauges. Monotonically increasing events - births, deaths, user interactions - use counters. Every metric write is lock-free (AtomicF64 under the hood), so the simulation loop pays almost nothing for instrumentation.

A new api endpoint exposes the metrics in Prometheus text format. The endpoint is gated behind a TTL that matches the scrape interval, so the gauge-collection logic only runs as often as Grafana actually asks for data.

Grafana Cloud dashboards

Grafana Cloud's free tier scrapes the api endpoint o man interval. On the Grafana side, I provisioned dashboards using Terraform, keeping dashboard definitions close to the code - server health (iteration rate, uptime, cache performance), simulation behavior (population dynamics, births and deaths by gender, species distribution, lifespan averages), and user metrics (entity interactions, controlled actions, active sessions).

Grafana Cloud metrics dashboard

The CORS dead end

The original plan was simple: point the browser directly at a public Grafana dashboard and pull stats from there. No server involvement needed for the client.

It didn't work. Grafana's public dashboard panel-query API does not set CORS headers on successful responses, and returns 401 on preflight OPTIONS requests. It works fine in curl but is completely unusable from a browser on a different origin. There is no configuration to fix this - it is a limitation of the public dashboard feature itself.

The authenticated proxy

With the browser ruled out as a direct Grafana consumer, I added same-origin proxy routes on the Rust server. Routes call Grafana Cloud's authenticated api , execute PromQL queries against the scraped metrics, and relay the results back to the browser as JSON.

What changed for the client

The UI looks the same. The home page ticker still cycles through population counts, birth rates, dominant species. The in-simulation overlays still show scoped and global statistics. The client code was rewritten to poll HTTP endpoints instead of holding a WebSocket connection, but the data it displays and the way it displays it did not change.

What changed is what's behind it. The numbers are now backed by a real time-series database with configurable retention. The server does less work - aggregation and history are Grafana's problem now. Adding a new metric means adding a gauge or counter on the server side; Grafana picks it up on the next scrape and it's immediately queryable in dashboards and in the proxy queries.

The result

The old stats tracker, the serializer, telemetry, the WebSocket endpoint - all gone. In their place: ~30 atomic metric writes per tick, a text-format scrape endpoint, and Grafana dashboards. The server is lighter. The data is richer. And I can finally see what happens in a world that runs for a week, not just what's happening right now.

See the live simulation metrics here.

A state of equilibrium

By Robin Dowling · a month ago

After months of collapse, an inherently unstable system has finally found equilibrium — sustaining a living symbiosis among plants and animals across more than 120 generations. No predefined species, no scripted behaviors. Everything in the simulation emerges from genetics, energy, and survival, and the system collapses more often than it survives. But a month ago, the right seed met the right settings and the world took hold.

More than 50 new species have branched off from the original population. Dense vegetation carpets the terrain. Flocks of animals migrate across the map in search of food and a place to breed. Grandparents, parents, children, and siblings all cohabitating the same world, trying to survive and procreate. The organisms that exist now are there because their ancestors found enough food, avoided enough predators, and located a mate in time. The plants that cover the ground are there because the soil had the right nutrients and the competition was not too fierce. All of it emerged on its own.

The odds were not good

I have written before about chasing equilibrium and coming up empty. The symbiosis required for a world like this — plants and animals all living off each other, depending on each other, varying a living ecosystem altogether — kept falling apart. I would tune a parameter and watch life bloom for a few thousand ticks, only for the whole system to unravel. Motiles would overcrop the sessiles. Sessiles would choke out the soil. Predators would eat themselves into extinction. Every path led to collapse.

The problem was never any single variable. It was all of them, entangled. Fifteen core parameters, all coupled, each one affecting every other. There was no gradient to follow, no knob to turn that reliably moved toward stability. The simulation would linger in an almost-stable state for what felt like long enough to be real, and then tip over a cliff.

Irreducible complexity

The state the world is in right now — 120 generations deep, dozens of species, a functioning energy cycle from sun to soil to plant to animal and back — is not a state I could have designed directly. It is irreducible. There is no shortcut to it. You cannot look at the starting conditions and predict whether they will produce a thriving ecosystem or a dead world. The only way to find out is to run the simulation and let every cause produce its effect, tick after tick, for millions of iterations.

This is what makes the achievement feel strange. I did not build this world. I built rules, and this world built itself. Whether it would work or not was genuinely uncertain until it did.

The anthropic observation

There is something unsettling about the parallel to the real universe. Life and thriving ecosystems are vanishingly scarce. We exist on a planet where the initial conditions happened to be just right — the right distance from a star, the right atmospheric composition, the right chemistry. The slightest variation and none of it works.

The same is true in the simulation. The initial state of the world is randomly generated at genesis. Most seeds produce worlds that collapse. The ones that survive do so because the random arrangement of organisms, soil composition, and spatial distribution happened to land in a configuration that could sustain itself through the brutal initial crash. The settings had to be just right, and there was no way to know in advance which settings those were.

I have run hundreds of seeds. Most die. This one lived. It lived because its genesis happened to place enough organisms close enough together, with enough genetic diversity, on terrain with enough nutrients, that the cascade of births and deaths and mutations converged on something self-sustaining rather than self-destroying.

What it looks like now

The world has a rhythm to it. Plant populations swell and recede with the energy budget. Animal populations follow, offset by a delay — the classic predator-prey lag. Species that were dominant 50 generations ago are gone. New ones have taken their place, adapted to niches their ancestors never occupied. Some species have split into two, pushed apart by accumulated mutations until they could no longer interbreed.

There are flocks. Animals of the same species cluster together, driven by fear and the safety of numbers. They migrate as a group, drifting across the map toward areas with more food. When a flock encounters another species, they either flee or hunt, depending on their size and hunger. It looks alive. Not designed-to-look-alive, but actually emergent from the rules.

What I learned

The thing that finally made equilibrium possible was finding the right combination of settings and the right seed. Every joule of energy in the world traces back to the sun. Every organism that exists paid for its existence with energy transferred from its mother. Every death returns nutrients to the soil. The mechanics had to be grounded in real physics, and the initial conditions had to land in the narrow window where those mechanics produce life instead of collapse.

The balance is not perfect. It never will be. Species will go extinct. Populations will crash. Some seeds will still produce dead worlds. But this one seed, running for a month in production, proved that the rules are capable of producing sustained life. That is enough.

It is enough because it means the search was worth it. Hundreds of seeds, countless iterations on the settings, and one world that found its footing and held on.

Real-Time State Sync Over WebSockets

By Robin Dowling · 2 months ago

Streaming simulation state to browser clients at 10 fps means every part of the pipeline has to handle the reality that clients fall behind, messages get dropped, and the world keeps moving regardless. Here's how the state sync works in Wilden, from the server's outbound queue to the client's state buffer.

Bounded drop-oldest outbound queue

Each connected client has a bounded outbound channel on the server, with a capacity of 8 messages. When the broadcast loop produces a new state message and the client's queue is full, the oldest undelivered message is evicted to make room for the newest one.

This means a slow client always converges toward the most current world state rather than falling further behind. The server never accumulates a growing backlog of stale state for a client that can't keep up, and a stalled client can never grow server memory.

The alternative — an unbounded queue or drop-newest — would either let a lagging client consume unbounded memory on the server, or force the client to process a long queue of outdated state before catching up to the present. Drop-oldest avoids both.

Delta state with forced full recovery

Most broadcasts are deltas: only the entity properties that changed since the last broadcast are sent, tracked per session. This reduces typical payload sizes by 70-90% compared to sending the full entity set every time.

But deltas are fragile. If a client misses a delta (because the outbound queue evicted it), the client's view drifts from reality. Two independent mechanisms recover from this:

Forced full state. Every 200 broadcasts, the server sends the complete entity set regardless of what the client already knows. Any accumulated delta drift is wiped clean. This is a timer-based safety net that requires no client cooperation.

Client feedback. When the client detects missing entities after applying a delta, it sends a feedback message to the server. The server clears its record of what that client knows and sends a full state on the next broadcast. This is a demand-driven recovery that kicks in immediately rather than waiting for the forced-full cadence.

Either mechanism alone would be sufficient. Having both means delta gaps are a transient visual glitch lasting at most a few hundred milliseconds, not a corruption bug that compounds over time.

Client-side state buffering

The client maintains a buffer of up to 5 pending state updates, keyed by iteration number. When a new state arrives, the buffer tries to find the next sequential iteration. If it's there, it's applied immediately. If it's not, the buffer waits — as long as it hasn't filled up.

When the buffer reaches capacity without the expected next iteration arriving, it falls back to the oldest available state. This handles the gaps created by the server's drop-oldest queue: the client sees iteration 5, then 7, with 6 never arriving. Rather than stalling, it applies what it has.

A world transition (server restart, new world) flushes all buffered states from the previous world. The first message from the new world always carries a full entity set, so the client rebuilds cleanly without waiting for the buffer to fill.

This buffering creates smoother visual transitions than immediately applying every state as it arrives. States that arrive slightly out of sequence get a chance to settle into order before being rendered. The behaviour ends up being similar to what you'd get with an unreliable transport like UDP — latest data wins, stale data is gone — but implemented at the application layer where the queue depth and fallback behaviour are tuneable.

The serialization pipeline

The wire format is a three-stage sandwich designed to minimize payload size:

  1. Custom minimization. Rich entity state is encoded into compact key-value strings with single-character keys. A field like worldId becomes a, coordinates become b and c. This strips the structural overhead that makes JSON verbose.

  2. JSON serialization + Brotli compression. The minimized structure is serialized to JSON (the interchange format between Rust's serde and the browser's JSON.parse), then Brotli-compressed. Brotli handles the remaining redundancy in the JSON encoding itself.

  3. MessagePack envelope. The compressed bytes are wrapped in a MessagePack binary map with a type discriminator and sent as a WebSocket binary frame. This eliminates UTF-8 framing overhead on the wire.

On the client side, the pipeline reverses: MessagePack decode, Brotli decompress, JSON parse, custom de-minimization. The result is a typed StateData object ready for the state buffer.

How the pieces fit together

The system is layered so each piece handles one concern:

The outbound queue handles backpressure. The delta protocol handles bandwidth. The forced-full cadence and client feedback handle recovery. The client buffer handles sequencing and smooth application. No single layer tries to solve everything, and each one is independent enough that tuning one (queue depth, forced-full interval, buffer size) doesn't break another.

The result is a state sync pipeline that handles thousands of entities across multiple concurrent clients, with typical per-update payloads around 1.5KB and peak bandwidth under 15KB/s — down from the 1.5MB per update and 10MB/s of the original always-full-state implementation.

Seeking equilibrium and balance in the world

By Robin Dowling · 2 months ago

I have been chasing equilibrium and balance in the world for some time. Not the kind of balance you find by turning a dial — the kind that emerges from a set of rules simple enough to fit on a page, and a world complex enough that those rules play out in ways I cannot predict. The goal is a simulation where life sustains itself indefinitely on any random seed, at any timescale, without my thumb on the scale. I have not found it yet. But the search has taught me more about ecology than any textbook I have read, and the failures have been more instructive than the partial victories.

A plant

The draining battery

For a long time I believed the simulation had a working energy cycle — sessiles feed on soil, motiles eat sessiles and each other, death returns nutrients to the ground. Then I started running worlds for longer, and they kept dying. I built a headless probe to measure what was actually happening, and the answer was bleak: what I had been calling a cycle was really a battery. Energy leaked out on every path — metabolic burn, 50% trophic loss on predation, only 15% of a corpse returning to soil. A closed system with those losses has exactly one stable state, and it is zero.

Worse, the one thing keeping the world alive was cheating. Every offspring received a fresh packet of energy minted from nothing — a hidden subsidy that could sustain predators indefinitely with no prey. That is nonsense in any real ecology.

The fifteen-dial trap

My first instinct was to tune my way out. Fifteen core parameters, all coupled. I ran an eleven-candidate sweep: lowering sessile fecundity delayed the boom but did not prevent it; raising corpse return rates made the boom bigger; reducing motile fecundity produced worse sessile blooms; increasing metabolic burn hit immobile sessiles hardest. Every lever moved the timing of the collapse, never the existence of it. The world was stuck between boom and slow death, and no constant-rate parameter could navigate between them.

Density-dependent brakes

If constant-rate parameters could not create a fixed point, maybe density-dependent ones could. I implemented Monod half-saturation brakes — reproduction scaled by local resources per capita, the kind of model ecologists have used since 1949. Sessiles braked by soil per plant; motiles by prey per predator. It produced coexistence for about a hundred thousand ticks, then motiles crept up, over-cropped the sessiles, and collapsed. I also tried a Holling type-II satiation mechanism — a digestion cooldown modelling predator handling time. Elegant in theory, insufficient in practice. All reverted. The lesson: you cannot regulate your way to stability in a system with no energy source.

Closing the mint

So I stopped tuning and started fixing the physics. The offspring energy mint was the hidden inflow — sized by the mother's fullness but never debited from her reserves. I made it a conserved transfer: the offspring's starting energy comes out of the mother, capped at what she can afford. Starving mothers cannot reproduce.

The results were dramatic. Seed 4, which had grown a plant-free motile population from 567 to 2,576 on minted energy, now decayed from 334 to 6. Motile irruptions stopped. This was by design worse in isolation — with the mint gone and no source added, every seed ran down faster. But it proved the trophic pyramid was closed at last. Nothing could grow without eating.

Adding the sun

Every real ecosystem is ultimately powered by sunlight, so I gave the simulation a sun. Plants capture a fixed budget of solar energy per unit of ground area each tick, shared among the sessiles on that patch — a crowded patch gives each plant less. That single detail is a density-dependent carrying capacity that falls out of the physics, no artificial cap required. It is also the Monod brake I tried to bolt on earlier, except grounded in real mechanics.

For the first time across every seed, the world stopped going extinct. Every joule now traces back to the sun, flows through plants into predators, and returns to the soil on death.

The smooth gate

Photosynthesis revealed a new problem: millions of half-starved plants reproducing at a trickle, collectively overrunning the world. The old three-bucket reproduction logic gave even the lowest energy category a nonzero chance. I replaced it with a continuous Hill curve — reproduction probability rising smoothly from zero to near-one, with a steep sigmoid at 50% energy. Sessile blooms roughly halved. Starvation now genuinely prevents reproduction.

The Allee threshold

With the economy grounded, I turned to calibrating the solar budget. At budget 0.3, all seeds survived 1.5 million ticks with zero resets — a first. But worlds leaned lopsided: oceans of plants, sometimes as few as seven motiles on a million-cell map.

That is when the real diagnosis landed: the Allee effect. In 1931, Warder Clyde Allee described how below a critical population density, per-capita growth drops to zero because organisms cannot find mates. My motiles must be within five position units of a compatible mate to reproduce, and can only detect mates within one map cell (125 units). At low density — thirty organisms on a million cells — encounters almost never happen. The empirical threshold is about 60 motile on the 1,000x1,000 map. Above that, mate-finding sustains itself. Below it, reproduction collapses.

This is not a bug. Real organisms that cannot find each other do not mate. The fix has to be on the density side: enough energy that the initial population crash does not drop motiles below the mate-finding threshold.

The solar budget as master knob

At budget 1.0, all five seeds survived 500,000 ticks with zero resets — motile populations still marginal (19-44) but alive. The solar budget controls carrying capacity: more sunlight means more plant energy, more food for grazers, more grazers surviving the initial crash, enough density for mate-finding to work. It is the one physically meaningful knob — the variable that determines how much life the world can support, just as it does on Earth.

But pushing the budget higher hit a ceiling. Each plant can only capture a fraction of its maximum energy from sunlight each tick — a cap designed to prevent solar-powered immortality. Budget 1.0 already saturates this cap, so tripling the sunlight had no effect. Raising the cap itself let more energy in but paradoxically made things worse: sessiles became energy-rich and boomed while motiles crashed harder and faster. The extra energy never reached the predators in time to prevent the initial population collapse.

A bigger world

The breakthrough came from the other direction: instead of pushing more energy into the same small world, I made the world bigger. A 3,000-by-3,000 map (nine times the area of the original) starts with 3,000-4,500 motile organisms instead of 300. Even after a brutal 90% initial crash, that leaves hundreds of survivors — well above the Allee threshold.

At 500,000 ticks, the results were the best I had ever seen. All five seeds survived with zero resets. Seed 1 bottomed at exactly 60 motiles and then recovered, climbing back to 89 with active pregnancies — the first genuine predator population recovery in the project's history.

The long horizon

Then I let them run to 1.5 million ticks, and every seed collapsed. Four out of five reset. The fifth survived as a single zombie motile — full energy, no mate, no future.

The pattern was identical across seeds: the initial crash stabilises around 50-100 motile for a few hundred thousand ticks, then a slow inexorable decline sets in. Organisms are well-fed — energy is not the problem. But as natural attrition thins the population (age, disease, predation), it crosses below the mate-finding threshold and never recovers. The 500,000-tick recovery on seed 1 was temporary. It peaked at 89 and then resumed declining. By 1.2 million ticks, every seed was down to single digits, full of energy, unable to find mates.

The bigger world bought time — 25 more real-hours before collapse compared to the small map — but it did not change the endgame. More organisms delays the Allee threshold crossing; it does not prevent it. The population leaks downward through natural mortality, and once it crosses below ~60, the feedback loop closes: fewer organisms means fewer encounters, fewer encounters means fewer births, fewer births means fewer organisms.

What I have learned

The simulation has taught me that equilibrium is not a place you arrive at by adjusting parameters. It is an emergent property of rules that create self-correcting feedback at every scale. My energy cycle is now physically grounded — every joule traces from the sun through plants to predators and back to soil. But the population dynamics still have a one-way ratchet: the Allee effect means any population that drifts below the mate-finding threshold is terminal, and nothing in the current rules pushes it back above.

The search continues. The options are larger worlds, smarter mate-finding, or mechanics that concentrate populations instead of letting them diffuse. But the deeper lesson is this: a world that can find its own balance needs rules that work at every population size, not just the comfortable middle. The edges — near-zero and near-carrying-capacity — are where equilibrium is won or lost.

Photosynthesis and the arithmetic of light

By Robin Dowling · 2 months ago

A while ago I wrote that the simulation had found "an eternal cycle of energy": sessiles feed on the soil, motiles feed on sessiles and each other, and death returns elements to the soil. It was a satisfying story. It was also, it turns out, not quite true.

Every world I ran would eventually die. Sometimes it took thirty hours, sometimes less, but the ending was always the same: a slow rundown to collapse, then a reset. I kept treating this as a tuning problem — nudging the fifteen-odd dials from that earlier post, hoping to catch the balance. It never held. So I finally built a proper measurement tool, a headless probe that runs a seed for hundreds of thousands of ticks and prints a census of the population, and I let it tell me what was actually happening instead of guessing.

A plant

The answer was uncomfortable. The "eternal cycle" wasn't a cycle at all — it was a battery slowly draining. Energy leaks out of the world on every path: organisms burn energy just to stay alive, only half of what a predator eats actually transfers (the rest dissipates), and only a fraction of a corpse returns to the soil. A closed system with that many leaks and no real inflow has exactly one stable state, and it's zero. Everything I'd been tuning was just adjusting the rate of the collapse.

Worse, the one thing keeping the world alive was cheating. Every time an organism reproduced, its offspring was handed a fresh packet of energy conjured from nothing. That was the hidden inflow propping the whole thing up — and because it was free, a population of predators could grow forever with no prey to eat, which is nonsense. A world with no plants should starve.

So I stopped tuning and started fixing the physics.

First, I closed the mint. An offspring's starting energy now comes out of its mother — a real transfer, capped at what she can actually afford, not a gift from the void. Immediately the nonsense stopped: a plant-free world of predators now decays instead of growing, because nothing can reproduce without paying for it out of energy it actually ate. This made every world run down faster — which sounds like a regression, but it was the point. With the fake inflow gone, I could finally see the true shape of the economy: all leaks, no source.

Then I gave the world a sun. Plants now capture a little sunlight each tick — photosynthesis, the thing every real ecosystem is ultimately powered by. I made the light a fixed budget per patch of ground, shared among the plants standing on it, so a crowded patch gives each plant less. That one detail turned out to be quietly important: it means the land has a natural carrying capacity built into the physics, without me imposing an artificial population cap. Finite sunlight over finite ground.

That was the moment the numbers changed. For the first time, across every random seed I threw at it, the world stopped going extinct over a full run. Every joule in the simulation now traces back to the sun, flows up through plants into predators, and returns to the soil on death. A real cycle, at last — not a story I told myself.

It is, I should be honest, not finished. Closing the mint left my predators too frugal to breed, so I loosened their reproductive cost and they recovered. Feeding the plants directly let them run away in dense blooms, so I changed how reproduction responds to hunger — a starving organism now genuinely can't breed, instead of breeding a tiny bit anyway, which used to let a million half-starved plants overrun the world. Each fix reveals the next imbalance. Right now the worlds survive, but they lean lopsided — oceans of plants, a thin scattering of grazers — and I'm still hunting the dial that gives a proper predator–prey balance rather than merely a persistent one.

Since writing the above, I have found the name for what is holding the predators back: the Allee effect. Warder Clyde Allee described it in 1931 — below a critical population density, per-capita growth drops to zero because organisms simply cannot find mates. My motile organisms must be within five position units of a compatible mate to reproduce, and they can only detect mates within one map cell (125 units). On a million-cell map with thirty scattered predators, those encounters almost never happen. The empirical threshold is about sixty motile organisms — above that, mate-finding sustains itself; below it, the population is alive but not reproducing. The solar budget controls how many predators survive the initial post-genesis crash, which determines whether the population lands above or below that threshold. It is the one physically meaningful knob: how much sunlight the world receives determines how much life it can support. I wrote more about this search in the next post.

But it feels different this time. I'm no longer turning dials at random on a system that can only lose. I'm shaping a world that has a source, a flow, and a floor — and asking it to find its own balance on top of that. Which, come to think of it, is the only kind of equilibrium worth finding.

Shared-Memory Parallelism: Why the Simulation Moved to Rust

By Robin Dowling · 3 months ago

The whole point of moving Wilden's backend from Node to Rust was a payoff I couldn't get on the old runtime: real shared-memory parallelism over a single live world, distributing state workloads across every core of the machine, without changing a single byte the browser client sees.

Why Node couldn't do this

The simulation is a few thousand organisms that live entirely in memory. Every tick, each one reads its neighbours' state (energy, position, mass, fear, what they're carrying), decides what to do, and acts. Keeping that state resident and hot is the whole game: it's what keeps per-tick latency low enough to broadcast state updates at 10 fps.

Node has one JavaScript thread. You can add worker_threads, but they don't share mutable memory. Each worker has its own heap, and to move data between them you either copy it (serialize, message-pass, deserialize) or confine yourself to a SharedArrayBuffer of raw bytes with no structured objects on top. For a workload whose hot path is "read arbitrary neighbours' rich state, mutate the world," both options are poison: serialization on every tick would dwarf the computation it's trying to parallelise, and a flat byte buffer throws away every type the simulation is built on. So in practice the tick stayed single-threaded, and "add more clients" literally meant "steal time from the simulation."

Rust removes the constraint. Threads share one address space; the borrow checker and the Send/Sync traits prove at compile time that concurrent access is safe, so I can hand many cores read-only references to the same live world and let them work in parallel with zero copying. That's the capability the migration was really for.

The shape of a tick

A tick has two halves:

  1. Decide (CPU-bound, and independent per organism so it splits cleanly across cores): every organism reads the world and produces a list of intents, such as "consume this soil," "fight that rival," "reproduce with this mate," "move here." No organism mutates anything but itself during decide.
  2. Commit (sequential, cheap): apply all the intents in a single deterministic order, resolving conflicts (two organisms eating the same soil, a kill that invalidates a later intent) by fixed rules.

Decide is where the cores go. Commit stays on one thread on purpose: its order is part of the contract.

Making the tick safe to parallelise

You can't just throw par_iter at a loop that reads and writes shared state mid-walk; you'd get a different result every run. Three changes made the decide phase order-independent by construction, so that parallel execution is provably identical to sequential:

  • Per-entity RNG streams. The old design drew from one global random generator in walk order, so any reordering changed every subsequent draw. Now each organism carries its own stream, seeded from splitmix(world_seed, num_id). A draw's result no longer depends on when in the tick it happens. (~32 bytes per organism, about 1.6 MB at 50k, cheap.)
  • Frozen tick-start reads. During decide, every cross-entity read sees the neighbour's value as it was at the start of the tick, not its half-mutated mid-tick value. Each organism still reads and writes its own live state (it exclusively owns itself), but it can never observe another organism's same-tick changes. This is built as a compact, read-only snapshot of the fields the decide phase consumes.
  • num_id-ordered commit. Intents are applied in a canonical order keyed by a monotonic birth sequence number, regardless of which thread produced them. Conflicts resolve the same way every time: soil over-draw clamps in actor order, kills are first-actor-wins, an already-pregnant mate drops later reproduction intents.

With those in place, a decide worker reads only frozen neighbour data plus its own slot, and emits intents into a per-worker buffer. Nothing it touches is contended. The proof I leaned on: run the decide walk backwards and the committed world is bit-for-bit identical. If order doesn't matter, threads don't either.

Distributing the work across cores

Once correctness no longer depends on how the work is partitioned, partitioning becomes a pure load-balancing and cache-locality choice, and the simplest option wins.

The decide phase is a rayon par_iter_mut over the organism store, sharded by store index (organisms.len() / k chunks), not by map position. Each worker takes a disjoint set of &mut Organism slots and shares one read-only neighbour view. Store-index sharding gives near-perfect load balance no matter how organisms cluster on the map, with zero rebalancing machinery. Intents fold into a per-worker buffer, then reduce into one before the sequential commit re-sorts them into canonical order, so the result is independent of how the work happened to shard across threads.

I deliberately rejected the fancier option. Spatial or checkerboard scheduling (coloring map cells so distant ones run concurrently) only pays off on much larger maps, couples correctness to geometry, and would still be scheduling-order-sensitive for RNG. The decide/commit model needs none of it. If very large worlds ever demand cache locality, spatial sharding can be reintroduced as an optimisation on top of the same correctness model.

Three pools, one core budget

A server core is wanted by three different consumers, and the trap is letting each grab "all cores":

Consumer Threads Work Parallel?
Engine 1 dedicated OS thread tick orchestration + sequential commit + snapshot publish No, commit order is the contract
Compute (rayon) available - reserved_cores the parallel decide phase Yes, this is the multicore tick
Transport (tokio) min(available - 1, 4) HTTP, website, all WebSocket I/O, per-session frame build Yes, but mostly parked in epoll_wait

These are sized as one partition, not three independent pools, because compute and transport contend in the same window: the transport plane builds tick N-1's frames while the engine computes tick N's decide. So a boot-time core budget resolves all three from a single number and logs what it picked. reserved_cores (default 2) is headroom held out of the compute pool for the engine thread and the transport plane's CPU burst, deliberately not one reserved core per tokio worker, because transport mostly yields and mild oversubscription is optimal there.

A second decoupling makes the client count irrelevant to tick latency: at the end of each tick the engine publishes an immutable Arc<SimSnapshot>, and every broadcast reads only that snapshot, never the live sim, never a lock the engine holds. Adding clients can no longer steal simulation time, which was the original sin of the Node design.

Performance

Two things are worth measuring separately: how expensive the tick itself is (compute), and how many clients a box can serve (transport).

Compute cost. A single-threaded tick is super-linear in organism count, because the dominant cost is the per-organism neighbour scan, which grows with neighbourhood density:

Tick @ entities 1 thread
1k ~5.8 ms
10k ~217 ms
50k ~4.0 s

At 50k a single-threaded tick is ~40× over the 100 ms frame budget. That gap is the entire reason the decide phase runs in parallel: spreading it across the compute pool is the lever for larger worlds. (These are single-thread reference numbers; the multi-core decide is still being tuned, so I don't yet have clean cross-core tick benchmarks to quote.)

Transport scaling. Here I do have measured cross-core data, because the snapshot decoupling means client count no longer steals tick time, so the only question is how many concurrent SCOPED clients a box holds at the 10 fps cadence. Measured over the full path (TLS + engine) at thewilden.com, with the load generator running off-box:

Machine vCPU decide clients frames/client/s % of 10 fps
c4a-standard-2 2 sequential 200 9.1 91%
c4a-standard-2 2 sequential 600 5.0 50%
c4a-standard-4 4 parallel 1,000 9.3 93%
c4a-standard-4 4 parallel ~1,800 6.9 69%

The 2-vCPU box (which collapses to a sequential decide) holds full cadence only to ~200 clients. The 4-vCPU box serves 1,000 concurrent clients at 9.3 frames/s and degrades gracefully past that rather than collapsing, because a bounded drop-oldest outbound queue means a slow client misses frames instead of blocking the engine. The jump is roughly 4× the throughput at a given load, because the parallel planes gain workers, not just clock.

Matching the machine

The deployment target is the GCP C4A (Axion, Arm Neoverse V2) performance series, c4a-standard-4 to -16 depending on world size. The reasoning is specific to this workload:

  • Dedicated physical cores, no SMT. On C4A every vCPU is a full physical core, so the engine, compute, and transport pools run on real hardware rather than sharing an execution unit via hyper-threading. That kills the per-tick jitter that SMT siblings cause on standard x86 VMs, and for a server that holds a tick rate for months, consistency beats peak clock.
  • Performance tier, not efficiency. C4A and N4A are both Axion; the difference is sustained throughput and memory bandwidth, which is exactly what a memory-bandwidth-bound neighbour scan cares about.
  • Titanium network offload. NIC packet processing is offloaded off the host CPU. Note, though, that TLS termination still runs on the guest, and the 1-core load test found CPU-side TLS handshakes to be the connection ceiling, which is precisely why more performance cores help under a connection burst.

The compute pool auto-sizes to available - reserved_cores, so scaling the world up is mostly a matter of picking a bigger machine: 4, 8, then 16 vCPU gives a 2-, 6-, then 14-worker decide pool, no config change.

Deploying it

The binary detects its allowance, not the host. std::thread::available_parallelism() honours cgroup CPU quotas and affinity masks on Linux, so a --cpus-limited container reports what it's actually been granted, and every pool sizes from that. The same image adapts to a 1-core box (where it runs a bit-identical sequential decide) or a 16-core box with no change, and parallelism.enabled: false even forces the sequential path explicitly. Deployment is a Docker image rolled out via Terraform to a long-lived Compute Engine instance, where an always-on server earns a committed-use discount that makes the performance tier affordable.

The payoff

Same seed, same world, on any core count: that's the determinism contract the parallel engine holds, verified by a golden-master digest and a statistical-faithfulness harness. The browser client never noticed; the wire format is byte-compatible throughout. What changed is that the simulation now uses the whole machine, the way it always wanted to, and the only reason it can is that Rust lets many threads share one live world safely. That capability was the entire argument for the migration, and this is it, working.

Plans as Execution Engines: Driving Hundreds of PRs From a Single Document

By Robin Dowling · 3 months ago

The hardest part of migrating Wilden's backend from Node to Rust wasn't the Rust code. It was continuity. The migration ran across four weeks and hundreds of work sessions, most of them driven by an AI agent that starts every session with no memory of the last one, many of them my own fifteen-minute check-ins from my phone. There was no internal context between sessions to rely on. Chat logs don't persist, memory doesn't transfer, and "I'll remember why I did that" is a lie you tell yourself once per week.

The thing that made it work was treating the plan document as a program: not a to-do list, but an execution engine and a state store that any session, human or AI, could pick up cold and run one safe step of.

The mental model: stateless worker, durable shared state

Think of it as a CPU and memory.

  • The session is the CPU. Ephemeral, no persistent registers. It executes one instruction, one unit of migration, and halts.
  • The plan doc is the program and the memory. It holds the instruction stream (what to do), the program counter (a "find the next task" router), and all persistent state: the status of every module, every decision made, every deviation recorded.
  • The repo is the heap, the durable artifacts the instructions mutate.

A session becomes one tick of a loop: read state → select next task → execute → verify → present → (human gate) → write state back → halt. Nothing of consequence lives in a session's head. If it isn't written in the plan or the repo, it didn't happen.

That single property, durable state plus a stateless worker, is what makes the whole process resumable (any session starts cold), parallelisable (independent lanes run at once), auditable (the plan's own git history is the migration's logbook), and deterministic (the same starting state routes every session to the same next task).

What goes in the plan

A plan that can run itself needs a few specific ingredients:

  • A cold-start preamble, the bootloader. Assumes zero prior context: where work happens, what infra already exists, how to confirm the environment is green, where to find the next task.
  • A tiny status vocabulary: [ ] not started, [~] in progress, [x] done. The set of all symbols across all modules is the migration's progress, at a glance.
  • The unit inventory, a dependency-ordered checklist of every module, each carrying its status symbol and a short landing note. This is the state store proper; the row is where a unit's state and history live.
  • A "find the next task" router, a deterministic protocol: scan for the first [ ], skip anything with an unmet dependency, and you land on exactly one next task with no judgement call.
  • An authoritative per-unit procedure, a fixed, numbered recipe every module follows, ending in a verification gate (type-check, both test suites, lint, build) that must pass before "done".
  • A decision log: cross-cutting choices recorded as short, numbered, immutable ADRs, so independent sessions stay consistent instead of contradicting each other.

The session lifecycle

One session, start to finish, is a loop with an inner review-until-clean loop and two terminal gates:

  1. Base on a fresh copy of the target branch and verify it before anything else.
  2. Find the next task via the router; confirm its predecessors are done.
  3. Implement and review, repeatedly. Run the per-unit procedure, then the full gate and a fresh critical review of the whole diff. Fix the real findings, refute the rest in one line, then re-run the gate and a fresh review. Repeat until a review pass finds nothing. "Tests pass once" is not the bar; "a fresh review finds nothing" is.
  4. Rebase on the latest target and re-run the whole gate.
  5. Gate: the automated checks must be green as a unit.
  6. Present: open the PR with what changed, why, and exactly how to test it by hand. Crucially, update the plan's own state in the same PR: flip the symbol, add the landing note, record any decision.
  7. Pause, then merge: a human manually tests against those instructions and merges. One PR per session. Then halt.

The next session repeats from step 1 against the new state. No handoff conversation is needed. The handoff is the committed plan plus the merged PR.

Fold the prompt into the plan

Here's the operational punchline. Because all standing context lives in the plan, the per-session kickoff prompt collapses to a single, identical, copy-pasteable line that only ever changes two pointers:

Work the plan in docs/plans/<plan>.md. Base everything on a fresh origin/<base> (verify HEAD == origin/<base> before doing anything) and follow the plan's session protocol. Then stop.

Anything a prompt would otherwise repeat every time (the setup, the find-next-task router, the gate, the review-until-clean loop, the rebase step, the write-back of state) lives in the plan, not the prompt. If you find yourself writing a long prompt, that's the signal: the missing instruction belongs in the plan, so the next session inherits it for free. A long prompt is per-session memory by another name, and per-session memory is exactly what this method abolishes.

This is what let me run the migration the way I actually did: offline sessions from my phone, kicking off that one line whenever I had a spare moment, then reviewing and manually testing the results at night. The plan remembered everything so I didn't have to.

Scaling out: many sessions at once

Because all durable state lives in the doc and the repo, never in a session's head, many sessions can run concurrently. The whole scheme rests on one safety rule:

A set of work is safe to run in parallel iff every member touches a disjoint set of units, stays in the same risk class, shares no unmet dependency, and makes at most one edit to any shared seam.

To make that hold, classify modules by blast radius: safe lanes (peripheral, low fan-in, parallelise freely), caution lanes (touch a sensitive seam, at most one per batch), shared seams (files many units co-edit, only one editor at a time), and hot units (the behaviour-changing endgame: a single serial track, never parallelised, but allowed to run alongside the safe lanes because the two never collide in code).

The plan doc is the one file every session co-edits, since each flips its own status row. Those collisions are expected and benign: you resolve them mechanically on rebase by re-applying your own row. You never design batches around avoiding plan-doc conflicts; you design them around the one rule.

Truth-maintenance: a stale plan is a bug

There's one discipline that holds the whole thing together: the plan must always read true. Because sessions act on what the doc says, inaccurate prose isn't untidiness, it's a defect that routes the next session into wrong work. So every session corrects stale claims on sight, updates the docs that the change touched, and drains any deferred work into a single durable backlog with a falsifiable "done" check. The same rule applies to deliberate non-changes: "we looked at X and chose not to touch it, because…" gets recorded too, so a future session doesn't re-litigate a settled decision.

What it prevents

Every failure mode of a long, multi-session migration maps to a missing ingredient:

  • Lost context between sessions: the state store plus the cold-start preamble.
  • Sessions contradicting each other: the decision log.
  • Premature, unsafe cutover: phases with hard gates and the bridge/dual-coverage pattern.
  • "Green once" shipping subtle bugs: the review-until-clean inner loop.
  • Acting on stale info: truth-maintenance.
  • Parallel sessions colliding: the one rule for safe concurrency.

The migration shipped across hundreds of pull requests, and I never once had to reconstruct "where was I?" The plan always knew. That's the real product here, not the Rust, but a way of working where the document is the source of truth and the sessions are interchangeable. The full playbook lives in the repo as a template you can lift for any large, incremental migration.

Same seed, same world. Same plan, same next step.