Backend Engineering · 1 of 2
Master synchronous (gRPC, REST) and asynchronous (event-driven, message queues) communication patterns in distributed systems.
In a monolithic application, services call each other via function calls (in-process). In a microservices architecture, services are deployed independently, forcing us to communicate over the network. This introduces latency, unreliability, and complexity.
The most common pattern for service-to-service communication.
Pros:
Cons:
When to Use: Simple request-response patterns, low-latency internal calls.
High-performance RPC framework using binary serialization.
Pros:
.proto files).Cons:
When to Use: High-throughput internal communication, real-time streaming.
// example.proto
service UserService {
rpc GetUser(UserId) returns (User);
rpc ListUsers(Empty) returns (stream User); // Server streaming
rpc CreateUser(stream UserCreate) returns (stream UserCreated); // Bidirectional
}
Services publish messages to a queue; consumers pull and process them.
Flow:
Producer -> [Message Queue] -> Consumer
(Persistent Storage)
Pros:
Cons:
When to Use: Background jobs, batch processing, high-volume events.
Unlike queues, Kafka retains messages for a retention period. Multiple consumers can read the same stream independently.
Key Difference:
Use Cases:
When a transaction spans multiple services (no distributed locks), use the Saga Pattern:
Each service listens to events and publishes new events.
1. Order Service: CreateOrder -> emit "OrderCreated" event
2. Payment Service: Listens to "OrderCreated" -> charges card -> emit "PaymentProcessed"
3. Inventory Service: Listens to "PaymentProcessed" -> updates stock -> emit "InventoryUpdated"
Pros: Loosely coupled, scalable. Cons: Hard to trace (spaghetti logic), difficult to reason about order of events.
A central orchestrator directs each service's actions.
1. Order Orchestrator -> "Charge Payment" command to Payment Service
2. Payment Service -> charges card, responds success
3. Order Orchestrator -> "Reduce Inventory" command to Inventory Service
4. Inventory Service -> updates stock, responds success
5. Order Orchestrator -> "Confirm Order" to Order Service
Pros: Clear flow, easy to debug, transaction boundaries explicit. Cons: Central orchestrator becomes a bottleneck/single point of failure.
If a step fails, execute compensating transactions to roll back previous steps.
Order Created -> Payment Charged -> Inventory Reduced
↓ (Out of stock!)
Rollback: Refund Payment
Rollback: Recreate Order
All services see the same data immediately. Requires distributed consensus (expensive).
Services may temporarily see different views, but converge over time. More scalable but harder to reason about.
Guarantees:
| Use Case | Pattern | Reason |
|---|---|---|
| Simple request-response | REST | Simplicity, low latency |
| High-throughput internal | gRPC | Performance, streaming |
| Distributed transaction | Saga (Choreography) | Decoupling, scalability |
| Audit trail / Analytics | Kafka | Retention, replay, many consumers |
| Background jobs | Message Queue | Durability, load smoothing |
Get an email when I publish the next chapter. No spam — just new technical deep-dives.
Comments
Share feedback or questions about this blog post.
No comments yet. Be the first to share your thoughts.