Scaling across instances

A single instance already goes a long way - the registry is proven to 10,000 concurrent connections (see below). When you outgrow one process, the Redis backplane lets you run many instances behind a load balancer with no change to your code.

Realtime::to_user("42").send(...) delivers to user "42"'s socket, but only if that socket is held by the same process that runs the send. With one instance, every socket is local, so nothing is needed. Behind a load balancer, that user's browser might be connected to instance A while the code that sends runs on instance B. The RedisBroker bridges that gap.

Single-instance performance

Before you reach for Redis, know what one process handles. The registry is proven to 10,000 concurrent connections on a single instance:

• A broadcast() to all 10,000 connections completes in ~16 ms.
• Under a registration/deregistration storm, a registry operation has ~5.3 ms latency at the tail.
• Broadcasts are non-starving: a slow or full connection never blocks the others.

Three design choices make that hold:

• spawn_blocking keeps any blocking work off the async runtime's worker threads.
• Snapshot-then-send resolves a target to a list of cloned senders under the registry read lock, then drops the lock before sending - so a broadcast never makes a concurrent register/deregister wait behind it.
• Bounded per-connection channels give correct back-pressure: a connection that can't keep up drops its own messages rather than stalling the broadcaster.

Cap the total with max_connections so a runaway client can't exhaust file descriptors.

How it works (multi-instance)

Every instance runs one background pump on a shared Redis channel (umbral:realtime:events):

1. Publish. When your code calls .send(...), the envelope is PUBLISHed to Redis instead of dispatched locally.
2. Subscribe. Every instance (including the one that published) SUBSCRIBEs and dispatches each received envelope to its own connection registry.

So a send on instance B reaches user 42 on instance A: B publishes, Redis fans out, A's pump delivers to the local socket. A connection is served exactly once (by the instance that holds it, via its subscription), never doubled. The pump reconnects with a fixed backoff if Redis drops.

Enabling it

The Redis backend is behind a cargo feature so single-instance apps carry no Redis dependency:

# Cargo.toml
umbral-realtime = { version = "0.0.1", features = ["redis"] }

Then point every instance at the same Redis:

use umbral_realtime::RealtimePlugin;
 
App::builder()
    .plugin(
        RealtimePlugin::default()
            .redis(std::env::var("REDIS_URL").unwrap())   // e.g. redis://10.0.0.5:6379/0
            .on_model::<Order, _, _>(|ev| async move {
                Realtime::to_group(format!("order:{}", ev.pk().unwrap_or(0)))
                    .send("changed", &ev.instance).await;
            }),
    )
    .build()?;

All instances must share one Redis (or a Redis cluster). Without the feature, .redis(url) isn't available; if a URL is set in a build that lacks the feature, the plugin logs a warning and falls back to the single-instance in-process broker rather than silently serving one instance.

What still needs care at scale

See also

• Transports for the push transports and the one-connection-per-browser model.
• Model subscriptions for the on_model / expose bridge used above.
• plugins/umbral-realtime/src/lib.rs (RedisBroker) for the pump implementation.
• The integration test plugins/umbral-realtime/tests/broker.rs (run with REDIS_URL=... cargo test --features redis -p umbral-realtime).