Roadmap › System Design — High-Level Design (HLD) from Basics to Case Studies › Module: Building Blocks
Lessons in this course (14)
7. Replication, partitioning, sharding and consistent hashing
Advanced★ Must knowLesson 7 of 14
Replication — copies of data on several nodes:
| Type | How | Trade-off |
|---|---|---|
| Leader–follower (primary–replica) | Writes to primary; replicas copy changes; reads can go to replicas | Replication lag → stale reads; failover needed |
| Multi-leader | Several nodes accept writes (multi-region) | Write conflicts to resolve |
| Leaderless (quorum) | Write to W nodes, read from R nodes, W + R > N for consistency | Tunable, complex |
Synchronous replication = no data loss on failover but slower writes; asynchronous = faster but may lose the last seconds. SQL Server: Always On availability groups; Azure SQL: geo-replication and read replicas.
Partitioning / sharding — split data across nodes so each holds a part:
| Method | How | Watch out |
|---|---|---|
| Range | By key range (A–M, N–Z; dates) | Hot ranges (all new orders go to the latest partition) |
| Hash | hash(key) % N | Resharding moves almost everything when N changes |
| Consistent hashing | Nodes and keys on a hash ring; key goes to the next node clockwise; virtual nodes balance load | Adding a node moves only ~1/N of keys — used by Dynamo, Cassandra, distributed caches |
| Directory/lookup | A service maps key → shard | Extra lookup, flexible |
Choose a shard key that spreads load evenly and keeps related data together (e.g. CustomerId for orders). Cross-shard queries and transactions become expensive — design around the main access pattern.