Choosing Rust for real-time event fan-out

The workload

Consider a service that sits between a high-rate upstream event feed and a set of dashboards. It normalizes incoming events, keeps a running aggregate, and fans the result out to every connected client inside a frame budget. The people watching expect the numbers to be correct now, not a second from now.

The interesting constraint in this class of system is not average throughput. It is bursts. Steady state is a trickle; then something happens upstream and the rate jumps by an order of magnitude for a minute or two. The system has to absorb that without dropping events and without stalling the steady-state path.

Why a garbage-collected runtime struggles here

A Node prototype will handle steady state comfortably. What tends to break under bursts is not throughput — V8 can serialize plenty of JSON — it is tail latency.

When the event rate jumps by 10x, GC pauses become visible downstream: hitches of a few hundred milliseconds, arriving during exactly the moments people are watching most closely. You can mitigate this — pre-allocate, pool buffers, keep closures out of the hot path — and it genuinely helps. But the floor is set by the collector, and for this workload the floor is not low enough.

Why Rust, Axum and Tokio

The reason is narrower than "Rust is faster":

• Predictable memory behaviour. There is no stop-the-world pause to design around. The requirement was never no pauses; it was no unpredictable ones.
• Tokio's scheduler matches bursty fan-out. A task per inbound connection, multiplexed across a thread pool, is the workload almost exactly. Nothing needs inventing.
• Axum stays out of the way. Tower middleware, routing, and WebSocket helpers. There is nothing magic in it and nothing to fight.

That combination lets the hot path be a tight, allocation-aware function while the cold paths — admin endpoints, health checks, config — stay ordinary handlers.

The shape of the architecture

upstream feed ──► ingest task ──► ring buffer (bounded mpsc)
                                       │
                                       ▼
                              normalizer task pool
                                       │
                            ┌──────────┴──────────┐
                            ▼                     ▼
                     aggregate actor        fan-out broadcaster
                     (single owner)         (broadcast::Sender)
                            │                     │
                            └──────────┬──────────┘
                                       ▼
                              connected clients

Three decisions worth calling out:

Shared mutable state gets a single owner. The running aggregate is the one piece of genuinely shared mutable state, and the simplest correct thing is to put it behind a single task that owns it. Updates arrive over an mpsc; reads reply over a oneshot. Conceptually this is slower than an RwLock; in practice it is usually faster and simpler, because there is no contention and no observer ever sees the state mid-update.

Fan-out is tokio::sync::broadcast. Each client subscribes; the aggregate task publishes. When a slow client falls behind, the channel reports Lagged and you send a snapshot to recover. You do not have to invent backpressure semantics — the channel already has them.

Easier to feel than to read:

Raise the rate. Slow a subscriber down and watch its buffer fill until it goes Lagged — that is the channel saying "you fell further behind than my buffer holds, here is what you missed, in aggregate." Speed it back up and snapshot recovery jumps the cursor to head. The sender never blocks on the slowest reader.

The ingest ring is bounded. A bounded mpsc means that if the upstream outpaces the normalizer you get backpressure instead of unbounded memory growth. Size the capacity to hold a couple of seconds of peak burst, then size the normalizer pool so that a couple of seconds is enough headroom.

The pitfalls worth knowing in advance

Cancellation safety. If a client disconnects mid-update, a naively written critical section can leave shared state half-modified. The fix is the usual one: compute the new value first, then swap it in atomically. This is a bug class you will not have encountered in a runtime without async cancellation, so it is worth looking for deliberately rather than waiting to meet it.

Observability is not free. A panic inside a Rust task is exactly as silent as an unhandled rejection in Node if nothing is wired up to report it. Structured logging with tracing from the first commit, not the tenth.

Treat inbound schemas as untrusted. Upstream feeds add fields without warning. Rust's strictness pushes you to define serde types early, which is good, but a strict deserializer turns a new field into a hard failure. Be permissive about unknown fields on inbound types and validate what you actually consume.

Iteration speed. Edit, compile, test is slower than in a scripting runtime. For steady-state work that is fine; for prototyping new handlers it is friction. A small "playground" binary that exercises the same handlers against canned JSON keeps the inner loop tight.

Would I choose it again

For this workload, yes. The reason is narrow and worth being honest about: the requirement was predictable latency under bursts, and the language provides that directly. If the requirement had been "serve a JSON API at a 100ms 99th percentile," staying on Node would have saved a meaningful amount of engineering time.

"Rust is faster" is a weak reason to pick Rust, because the garbage-collected version is usually already fast enough. "Rust gives me a tail-latency profile I can reason about" is a real one. Pick it for that.