Roadmap › System Design — High-Level Design (HLD) from Basics to Case Studies › Module: Building Blocks
Lessons in this course (14)
8. Queues, event streaming and asynchronous processing
Intermediate★ Must knowLesson 8 of 14
| Message queue | Event stream (log) | |
|---|---|---|
| Examples | RabbitMQ, Azure Service Bus, SQS | Kafka, Azure Event Hubs, Kinesis |
| Consumption | Message removed after processing; competing consumers | Messages kept for days; each consumer group tracks its offset; replay possible |
| Ordering | Per queue/session | Per partition |
| Use | Background jobs, commands, decoupling services | Analytics, event sourcing, IoT, activity tracking, CDC |
Benefits: decoupling (producer doesn't wait for the consumer), load levelling (absorb spikes), reliability (retry, dead-letter queue), fan-out (many subscribers). Design consumers to be idempotent (at-least-once delivery). Use the outbox pattern to publish events reliably with database changes.