Cloud Tech by Victor
System DesignAdvanced

Database Sharding

Why a monotonically increasing shard key like a sequential ID or timestamp routes every new write to the same shard, and why hashed sharding fixes that distribution problem at the cost of range queries no longer targeting a single shard.

Updated 2026-07-244 min read

Overview

Sharding scales a database horizontally by splitting one logical dataset across many physical shards, and the shard key, the field (or fields) used to decide which shard a document lives on, is the single most consequential choice in the whole design. A low-cardinality shard key concentrates most data onto too few shards; a monotonically increasing key (a sequential ID, a timestamp) is worse under range-based sharding specifically, every new write's value is higher than everything already inserted, so all new writes land on whichever single shard currently owns the highest range, a hot shard problem that gets worse the more the cluster grows, not better. Hashed sharding fixes that distribution problem by scattering even monotonic keys across shards via their hash, at a real, structural cost: a range query on the original key can no longer target one shard, since the corresponding hashes are scattered everywhere, and becomes a broadcast query across the whole cluster instead.

Quick Reference

Sharding strategyDistributionRange query locality
RangedBy actual shard-key value rangesGood, a range query can target one shard
HashedBy hash of the shard-key valuePoor, range queries broadcast across shards
Shard key problemCauseResult
Low cardinalityFew distinct values, skewed real-world frequencyMost data lands on one shard
Monotonically increasingEvery new value is higher than the lastAll new writes hit one shard

Syntax

javascript
// Ranged sharding - good for range queries, bad for a monotonic key
sh.shardCollection("mydb.events", { createdAt: 1 });

// Hashed sharding - even distribution, even for monotonic keys
sh.shardCollection("mydb.events", { userId: "hashed" });

Examples

javascript
// A sequential/monotonic shard key concentrates every new insert
// on whichever shard currently owns the highest range.
sh.shardCollection("mydb.users", { userId: 1 });
// All new inserts land on one shard, the classic hot-shard case.

// Fixed with hashed sharding on the same field.
sh.shardCollection("mydb.users", { userId: "hashed" });

Hashed sharding trades away range-query locality

Fixing a hot shard with hashed sharding means a query filtering a range of the original key (recent timestamps, a sequential ID range) can no longer be routed to one shard, it becomes a broadcast query across every shard in the cluster. This is a real trade-off, not a free fix.

Visual Diagram

Common Mistakes

  • Choosing a shard key with high theoretical cardinality but skewed real-world value frequency, still producing a hot shard despite "enough" distinct values existing.
  • Using a sequential ID or timestamp as a range-sharded key without recognizing it guarantees all new writes concentrate on one shard.
  • Switching to hashed sharding to fix a hot shard without accounting for the resulting loss of range-query locality across the cluster.
  • Assuming a shard key choice is easily reversible; resharding an existing large collection is documented as resource-intensive, not a quick fix.

Performance

  • A hot shard bottlenecks the entire cluster's write throughput on one node's capacity, regardless of how many total shards exist, adding more shards doesn't help until the key itself is fixed.
  • Hashed sharding's range-query cost is real and structural: a query that would hit one shard under ranged sharding becomes a broadcast across every shard under hashed sharding, a direct latency and load cost on that query pattern.

Best Practices

  • Choose a shard key with genuinely even real-world value distribution, not just high theoretical cardinality.
  • Avoid monotonically increasing fields as a ranged shard key; use hashed sharding specifically for fields that increase this way.
  • Weigh write-distribution needs against read-pattern needs before choosing ranged versus hashed, they optimize for different things.
  • Treat shard key choice as a decision to get right upfront, resharding later is documented as resource-intensive, not a routine operation.

Interview questions

What is a shard key, and why does a low-cardinality shard key (few distinct values) cause a hot shard?

A shard key is the field (or fields) a sharded database uses to decide which shard each document or row actually lives on. If that field has few distinct values, say `country`, and the real data is skewed (80% of users in one country), the vast majority of documents route to the same shard regardless of how many shards exist in the cluster, overwhelming it with disproportionate read/write load and storage while other shards sit comparatively idle. Cardinality alone doesn't guarantee even distribution either, the values also need to actually occur with reasonably even frequency in the real data, not just theoretically have many possible values.

Why does using a monotonically increasing field (a sequential ID, a timestamp) as a shard key concentrate all new writes onto one shard, even with range-based sharding across many shards?

With range-based sharding, chunks are assigned contiguous ranges of shard-key values, and a monotonically increasing key means every new document's value is higher than every previously inserted one, so all new inserts land in whatever chunk currently owns the highest range, which lives on one specific shard. Every other shard, holding older, lower-valued ranges, receives none of the new write traffic at all, the exact "hot shard" problem, all insert load concentrated on a single shard regardless of how many total shards the cluster has.

What does hashed sharding trade away in exchange for fixing the monotonic-key hot-shard problem?

Hashed sharding computes a hash of the shard key value and assigns chunks by hash range instead of by the raw value, which scatters even monotonically increasing keys roughly evenly across shards, since consecutive input values hash to essentially unrelated output values. The trade-off is range-query locality: a query filtering a range of the original shard key values (like "the last 24 hours" on a timestamp key) can no longer be routed to one contiguous set of shards, because the corresponding hashed values are scattered unpredictably across the whole cluster, turning what would have been a single-shard query under range sharding into a broadcast query touching every shard.

References

ShareXLinkedInReddit
Was this page helpful?
Suggest an improvement