Why Your Payment Backend Needs a Dedicated Real-Time Layer (And How We Built One)

You know that moment when you order food on Zomato, you pay, and the delivery partner's phone instantly lights up with "Payment Received"? It feels seamless. Almost magical. But behind that two-second moment, there's a surprisingly complex system making it happen.

I recently built a payment processing prototype and ran into a problem that every team building real-time payment experiences eventually hits: the thing that processes payments should not be the same thing that delivers notifications. Let me walk you through why, and how splitting these into two separate systems made everything better.

Let's start with the obvious approach. You build a payment backend. It talks to Razorpay, handles webhooks, updates the database, and when a payment goes through, it pushes a message to the delivery partner over a WebSocket.

This is what I built first. And honestly, it works. For a prototype. For ten users. Maybe even a hundred.

Here's where it starts to fall apart.

The payment API is an HTTP server — it handles requests, talks to the database, and responds. WebSocket connections, on the other hand, are long-lived. They sit there, open, waiting. Every delivery partner with the app open is holding a persistent connection to your server. Now your payment API is doing two fundamentally different jobs: processing short-lived HTTP requests and babysitting thousands of long-lived socket connections.

The notification part was also naive. When a payment was confirmed, the server would broadcast the event to every connected WebSocket client. Every single one. It didn't care who the payment was for. The client-side app had to figure out "is this event meant for me?" That's wasteful, insecure, and just bad design.

So I decided to separate things.

The second system I built is inspired by something CRED (the fintech company) built internally called Propeller. The idea is simple but powerful: build a standalone service whose only job is managing persistent client connections and routing events to the right user, on the right device.

It doesn't know what a payment is. It doesn't care about orders or Razorpay or UPI. All it knows is: "I have a bunch of connected clients, and when someone tells me to send a message to client X, I deliver it."

Think of it like a postal service. The postal service doesn't write letters. It doesn't know what's inside the envelope. Its entire job is making sure the letter reaches the right address.

Here's the full flow of what happens when you tap "Pay" on a food delivery app, with both systems doing their part.

Let me explain why having Propeller as a separate service is a fundamentally better design.

One detail that's easy to overlook but critical to get right: keeping connections alive.

Mobile networks are unreliable. Phones go in and out of coverage. Apps get backgrounded. Without actively monitoring connection health, you end up with ghost connections — sockets that look open on the server but have no living client on the other end. Events sent to ghost connections just vanish.

Propeller handles this with a keepalive mechanism. For WebSocket clients, the server periodically sends a Ping frame. If the client doesn't respond with a Pong within a timeout window, the connection is considered dead and cleaned up. For gRPC streaming clients, it monitors idle streams and disconnects them after a configurable timeout.

This ensures the system always has an accurate picture of who is actually online and reachable.

Every connection to Propeller requires authentication via JWT tokens. The token contains the client ID and tenant ID — Propeller knows who this user is and which org they belong to. There's no way for one user to subscribe to another user's channel.

For backend services publishing events (like the payment worker), Propeller requires an additional API key. This creates a clear separation:

A compromised client token can never be used to inject fake payment confirmations.

There's one more piece that ties everything together. The payment worker runs a reconciliation process every five minutes. It scans the database for payments still in PENDING state beyond a reasonable time window and resolves them. Maybe a webhook was lost. Maybe the gateway had a hiccup. Whatever the reason, reconciliation catches these edge cases and ensures no payment gets stuck in limbo forever.

In the real world, distributed systems don't behave perfectly. Webhooks get lost. Network calls fail. Redis might have a brief outage. The reconciliation worker acts as a safety net, ensuring eventual consistency even when individual components hiccup.

The key takeaway: real-time payment notifications are not a feature you bolt onto your payment backend. They're an infrastructure problem that deserves its own dedicated service.

By separating the "processing" from the "pushing," you get a system where each part can scale, fail, and evolve independently. The payment backend stays lean and focused on handling money safely. The push layer stays lean and focused on delivering messages fast and to the right person.

This is the same pattern that companies like CRED, Zomato, and others use at scale. And once you have a Propeller-like service running, it's not just for payments. Order updates, chat messages, live tracking, surge pricing alerts — any event that needs to reach a user in real time can flow through the same infrastructure.

Build the plumbing right, and everything else becomes easy.