HardSystem design
How do you design a chat application like WhatsApp or Slack?
- Technology:
- System Design
- Experience:
- Senior
Quick answer
Clients keep a persistent WebSocket connection to stateless chat servers; messages are stored durably, routed to the recipient’s server through a pub/sub layer, and delivered as push notifications when the recipient is offline.
Problem
Design a messaging service with one-to-one and group chats, online presence and message history.
Functional requirements
- Send and receive text messages in real time in one-to-one and group chats.
- Store message history and sync it across a user’s devices.
- Show delivered and read status.
- Show whether a contact is online.
- Notify offline users with push notifications.
Non-functional requirements
- Scalability: millions of concurrent connections.
- Availability: messaging should keep working during partial failures.
- Performance: messages delivered in well under a second to online users.
- Reliability: no message loss; order preserved within a conversation.
High-level architecture
- Clients connect over WebSockets to chat servers behind a load balancer. Each server holds many connections; a connection registry (for example in Redis) maps user and device to server.
- When a message arrives, the chat server assigns it a per-conversation sequence number, writes it to the message store, and acknowledges the sender.
- It then publishes the message to the recipients’ servers through pub/sub. Each server pushes it down the matching connections; if a recipient has no connection, a push notification is queued.
- On reconnect, a client asks for messages after the last sequence number it has, which also covers missed messages.
Main components
- Load balancer
- Distributes WebSocket connections; supports long-lived connections.
- Chat servers
- Stateless apart from open connections; handle sending, acks and delivery.
- Connection registry / presence
- In-memory store of which server each online device is connected to, with heartbeats for presence.
- Pub/sub or message queue
- Routes messages between chat servers and to the push notification service.
- Message store
- Write-heavy store partitioned by conversation id, for example a wide-column database.
- API service
- Regular HTTP endpoints for login, contacts, conversation lists and history pagination.
Database design
- messages(conversation_id, sequence, message_id, sender_id, body, created_at) partitioned by conversation_id and clustered by sequence.
- conversations(id, type, created_at) and conversation_members(conversation_id, user_id, last_read_sequence).
- Read receipts are derived from last_read_sequence rather than stored per message.
Scaling
- Add chat servers horizontally; the registry tells others where each user is connected.
- Partition message storage by conversation so one chat’s history lives together.
- For very large groups, fan out on read (members fetch from the conversation) instead of pushing to every member.
- Rate limit senders to protect against spam and abuse.
Trade-offs
- WebSockets give real-time delivery but make servers stateful in terms of connections; registries and graceful reconnects are required.
- Ordering per conversation (not globally) keeps the system scalable.
- Storing messages before delivery adds a little latency but guarantees durability.
Discussion
A chat system needs low-latency, ordered delivery between users who may be online on several devices or offline entirely. The main design questions are how clients stay connected, how a message finds the server holding the recipient’s connection, and how messages are stored and synced.
Start with one-to-one messages, then extend to group chats, read receipts and presence. State your assumptions on scale early, because they decide the storage choice.
Key points
- WebSockets for real-time delivery
- Connection registry to route messages between servers
- Per-conversation sequence numbers for ordering and sync
- Push notifications for offline users
Follow-up questions
- How do you design a notification system?
- How would you add end-to-end encryption?