Everyone Said 'Just Go Serverless. ' I Ran a Long-Lived Node Process — Here's What That Bought
When a product requires thousands of persistent WebSocket connections, a single long‑lived Node process can be far more efficient than serverless functions. The article explains how in‑process state, loop‑based fan‑out, and backpressure handling reduce costs and improve performance, while also outl…
When you build a real‑time dashboard that pushes telemetry to thousands of users, the most obvious choice is often to deploy everything on a serverless platform. Serverless shines for short, stateless request/response workloads, but it struggles when the core of the product is a long‑running connection. In this article the author explains why a single long‑lived Node process can outperform serverless for this scenario, and what trade‑offs come with that decision.
Why Serverless Is Not a One‑Size‑Fit for Real‑Time
Serverless pricing is based on the number of invocations and the time a function stays warm. The model assumes that a function is invoked, does one job, and exits. That model is perfect for REST APIs, scheduled jobs, or image processing, but it breaks when the product’s value is tied to a persistent connection. A real‑time dashboard keeps a socket open for each user, sending updates as soon as they arrive. In that context, the cost of maintaining thousands of idle connections in a serverless gateway is linear in the number of users, even when nothing is being transmitted.
By contrast, a long‑lived process can hold all those sockets in memory for free. The operating system only charges for the file descriptors and a small amount of RAM, while the application logic can keep the connections alive without paying per‑message fees.
What a Long‑Lived Process Gives You
There are several concrete benefits to keeping the process alive:
- In‑process state – A subscription registry, a per‑user cache, and rate‑limit windows can live in memory. Every message no longer needs a round trip to Redis; it becomes a simple hash lookup.
- Loop‑based fan‑out – Sending one update to 5,000 subscribers is just a single iteration over a Map. In a serverless gateway you would incur 5,000 separate invocations, each billed and each with cold‑start latency.
- Backpressure handling – The socket layer can signal when a client is not keeping up. A stateless function never sees the same client twice, so it can’t react to backpressure and may overwhelm downstream services.
- Warm resources – Connection pools, compiled serializers, and prepared database statements are created once at boot. In a serverless function they would be re‑established on every cold start, adding latency for the first user that connects.
- Ordering guarantees – Messages from a single client arrive in order within one process. Across stateless invocations you would need a queue and sequence numbers to preserve order.
The Extra Responsibilities You Must Take On
Running a long‑lived process is not a free lunch. The developer must now own several operational tasks that serverless would otherwise handle:
- Deployment without dropping connections – A draining strategy is required: stop accepting new connections, let existing ones finish, then shut down the old instance.
- Scaling decisions – You decide when to spin up or down instances. Auto‑scaling is still possible, but the logic lives in your code rather than the platform.
- Health checks and restarts – A crash kills all open connections, so you need to detect failures and restart the process without losing state.
- Memory discipline – Leaks that would be harmless in a short‑lived function become critical because the process runs for hours or days.
These responsibilities amount to roughly a week of infrastructure work that you pay for once, but they can dramatically reduce per‑user cost and improve latency for every session.
When to Stick With Serverless
The author notes that serverless remains the best choice for:
- REST APIs that wrap the real‑time core.
- Scheduled jobs or cron tasks.
- Webhook receivers that process occasional bursts.
- Image or file processing that is inherently stateless.
In practice the optimal architecture is a hybrid: a long‑lived Node process for the WebSocket layer, and stateless functions for everything else. The boundary is drawn where state stops being a core requirement.
Takeaway: Match Architecture to Product Constraints
When deciding between serverless and a long‑lived process, start by asking: what is the single constraint that defines the product? If the answer is “connections must stay open,” then the default serverless answer is likely wrong. The cost of the extra operational work is outweighed by the performance and cost benefits for the user. If the constraint is “stateless request/response,” then serverless wins.
By keeping this decision framework in mind, you can avoid the trap of blindly following the default and instead make an architecture that truly serves the product’s needs.
Why it matters
Choosing the right runtime for a real‑time dashboard directly impacts user experience and operational cost. A long‑lived process can deliver instant updates with lower latency, while serverless may introduce hidden costs and complexity.
Key points
- Serverless excels at stateless request/response, not persistent connections.
- Long‑lived Node keeps state in memory, eliminating per‑message Redis calls.
- Loop‑based fan‑out is cheaper than 5,000 serverless invocations.
- Backpressure can be handled in a persistent process but not in a stateless function.
- Deploying a long‑lived process requires draining, scaling, and memory discipline.
- Hybrid architecture: use long‑lived for WebSockets, serverless for other services.
Frequently asked questions
What is the main cost advantage of a long‑lived process for WebSockets?
It eliminates per‑message billing and reduces latency by keeping sockets open in memory, costing only a few file descriptors and RAM.
How do you handle scaling with a long‑lived Node process?
You implement your own scaling logic, often using a load balancer that routes new connections to healthy instances and drains old ones.
Can I still use serverless functions for other parts of my app?
Yes, a hybrid approach is common: keep the WebSocket layer long‑lived, and use serverless for REST APIs, jobs, and webhooks.
