Search
29 results for “distributed-systems”
Search results
Consistent Hashing
Why placing hosts and keys on a hash ring means adding or removing one host out of N only remaps roughly 1/N of the keys, instead of the near-total remapping a plain modulo hash would force on every single change.
Database Sharding
Why a monotonically increasing shard key like a sequential ID or timestamp routes every new write to the same shard, and why hashed sharding fixes that distribution problem at the cost of range queries no longer targeting a single shard.
Idempotency in Distributed Systems
Why Stripe returns the exact same response, error included, for a reused idempotency key instead of retrying the operation, and why reusing that same key with different parameters is treated as an error, not a new request.
Load Balancers
How load balancers distribute traffic across servers, algorithms, health checks, Layer 4 vs Layer 7, and the failure modes that show up at scale.
Message Queues & Event-Driven Architecture
Why a visibility timeout, not a delete, is what actually protects a message from being processed twice, and why "at-least-once delivery" means your consumer has to handle duplicates even when everything is configured correctly.
Rate Limiting Algorithms
How the token bucket algorithm AWS API Gateway actually uses separates a steady-state rate from a burst allowance, and why a request only fails once the bucket is genuinely empty, not the instant the average rate is exceeded.
The CAP Theorem
Why partition tolerance isn't actually optional for a distributed system, and why that leaves only a choice between consistency and availability once a real network partition happens, not a free choice among all three properties all the time.
Why does a naive `hash(key) % N` scheme for distributing keys across N servers fall apart the moment a server is added or removed?
With plain modulo hashing, the server a key maps to depends directly on the current value of N, since almost every key's `hash(key) % N` result changes the instant N changes to N-1 or N+1, even though the underlying hash values themselves didn't change at all. That means adding or removing a single server can remap the overwhelming majority of keys to different servers simultaneously, which for a cache means a massive wave of cache misses, and for a sharded store means a massive, unnecessary data-migration event, triggered by a change to just one server out of many.
How does consistent hashing (a ring hash) avoid that near-total remapping problem?
Instead of computing `hash(key) % N`, both hosts and keys are hashed onto positions on a fixed conceptual ring (typically the hash function's full output range), and each key is assigned to the next host found by walking clockwise from the key's position. Removing a host only affects the keys that were mapped to that specific host's section of the ring, they get reassigned to the next host further along, while every other key on the ring, owned by a different host's section entirely, is completely unaffected. For a ring hash across N hosts, adding or removing one host affects only about 1/N of the total keys, not nearly all of them.
Why does a real ring hash implementation give each host many positions on the ring instead of just one, and what problem would a single position per host cause?
A host thrown onto the ring at just one point can end up, purely by chance, owning a disproportionately large or small arc of the ring if the hash values happen to land unevenly, since with few points there's no averaging effect smoothing out the randomness. Assigning each host many positions on the ring, scaled by that host's intended weight, so a double-weight host gets roughly twice as many ring entries as a single-weight one, averages out that randomness across many smaller arcs per host, producing a much more even overall traffic distribution than a single coin-flip-like placement per host would.
What is a shard key, and why does a low-cardinality shard key (few distinct values) cause a hot shard?
A shard key is the field (or fields) a sharded database uses to decide which shard each document or row actually lives on. If that field has few distinct values, say `country`, and the real data is skewed (80% of users in one country), the vast majority of documents route to the same shard regardless of how many shards exist in the cluster, overwhelming it with disproportionate read/write load and storage while other shards sit comparatively idle. Cardinality alone doesn't guarantee even distribution either, the values also need to actually occur with reasonably even frequency in the real data, not just theoretically have many possible values.
Why does using a monotonically increasing field (a sequential ID, a timestamp) as a shard key concentrate all new writes onto one shard, even with range-based sharding across many shards?
With range-based sharding, chunks are assigned contiguous ranges of shard-key values, and a monotonically increasing key means every new document's value is higher than every previously inserted one, so all new inserts land in whatever chunk currently owns the highest range, which lives on one specific shard. Every other shard, holding older, lower-valued ranges, receives none of the new write traffic at all, the exact "hot shard" problem, all insert load concentrated on a single shard regardless of how many total shards the cluster has.
What does hashed sharding trade away in exchange for fixing the monotonic-key hot-shard problem?
Hashed sharding computes a hash of the shard key value and assigns chunks by hash range instead of by the raw value, which scatters even monotonically increasing keys roughly evenly across shards, since consecutive input values hash to essentially unrelated output values. The trade-off is range-query locality: a query filtering a range of the original shard key values (like "the last 24 hours" on a timestamp key) can no longer be routed to one contiguous set of shards, because the corresponding hashed values are scattered unpredictably across the whole cluster, turning what would have been a single-shard query under range sharding into a broadcast query touching every shard.
A client sends a payment request, the network times out before a response arrives, and the client retries with the same idempotency key. What actually happens on the server?
If the original request already completed (successfully or even with an error like a 500) before the retry arrives, the server returns the exact same result it returned (or would have returned) the first time, the same status code and response body, without re-executing the underlying operation, so the customer isn't charged twice just because the client never saw the first response. This is the entire point of an idempotency key: it lets a client safely retry an operation whose actual outcome it's genuinely uncertain about (did the first request even reach the server? did it complete before the timeout?) without that retry risking a duplicate side effect.
Why does reusing the same idempotency key with different request parameters return an error instead of just processing the new parameters?
An idempotency key is a promise that a specific, exact operation happened once; if the same key showed up with different parameters, honoring the new parameters would silently violate that promise; either the original operation's recorded result no longer describes what the key represents, or the client made a mistake by rIeusing a key it should have generated fresh for a genuinely different request. Rejecting the mismatched reuse as an error, rather than guessing which parameters were "correct" or silently processing the new ones, surfaces that client-side mistake immediately instead of masking it.
Why are idempotency keys typically associated with state-changing requests like POST, and are they ever relevant to DELETE?
GET is defined to have no side effects at all, retrying it any number of times simply returns the current state and changes nothing, so there is no duplicate-side-effect risk an idempotency key could protect against, GET genuinely doesn't need one. DELETE is idempotent in the narrow sense that deleting an already-deleted resource is a no-op or a consistent "not found," but whether an idempotency key is relevant to it is API- and version-specific, not settled by the HTTP verb alone: a DELETE can still trigger complex, non-idempotent side effects (a refund, a cascading cleanup, a billing adjustment), and some APIs explicitly accept an idempotency key on DELETE for exactly that reason. Idempotency keys exist to make an operation's retry semantics explicit and safe regardless of whether the underlying operation is inherently idempotent, checking the specific endpoint's documented contract matters more than assuming based on the verb.
What is the difference between Layer 4 and Layer 7 load balancing?
A Layer 4 load balancer operates at the transport layer, routing based on IP address and port without inspecting the actual request content; it is fast and protocol-agnostic but cannot route based on things like URL path or headers. A Layer 7 load balancer operates at the application layer, so it can inspect HTTP requests and route based on path, host header, cookies, or content type, more flexible (path-based routing, A/B testing, session affinity by cookie) but with more per-request processing overhead.
How does a load balancer detect and handle an unhealthy backend server?
Via health checks, periodic requests (a lightweight HTTP endpoint like /healthz, or a TCP connection check) sent to each backend on an interval. A server that fails a configurable number of consecutive checks is marked unhealthy and removed from the rotation; it is added back only after passing a configurable number of consecutive successful checks, which avoids flapping a server in and out of rotation on a single transient failure.
Why is round robin sometimes a bad load-balancing algorithm choice?
Round robin assumes every server and every request is roughly equal cost, which is not always true, if backend servers have different capacities, or requests vary wildly in cost (a cheap health check vs. an expensive report generation), round robin can overload a server that happens to be mid-way through several expensive requests. Least-connections or weighted algorithms account for current load or server capacity instead of blindly cycling through the list.
What is a visibility timeout, and what problem does it solve that simply having multiple consumers poll a queue would create?
When a consumer receives a message, the queue doesn't delete it immediately, it starts a visibility timeout during which that message is hidden from other consumers, so a second consumer polling the same queue won't also pick up and process the same message concurrently. The consumer is expected to finish processing and explicitly delete the message before the timeout expires; if it does, the message never reappears. Without this mechanism, multiple consumers competing for messages would routinely double-process the same one, exactly the race condition visibility timeouts exist to prevent.
A consumer receives a message, starts processing, but crashes before deleting it. What happens to that message, and why is this actually the desired behavior?
Once the visibility timeout expires without the message being deleted, it automatically becomes visible again in the queue and can be picked up by the same or a different consumer for another processing attempt. This is deliberate, not a bug: the alternative (a crashed consumer's message vanishing permanently) would silently lose data, whereas reappearing after timeout guarantees eventual processing at the cost of a possible duplicate attempt, which is exactly the trade-off "at-least-once delivery" describes. Setting the visibility timeout too short causes premature, unnecessary reprocessing of messages still legitimately being worked on; too long delays legitimate retries after a real crash.
What does "at-least-once delivery" actually guarantee, and what does it not guarantee, and why does that mean a consumer must be idempotent?
At-least-once delivery guarantees a message will eventually be delivered and processed at least one time, it does not guarantee exactly one delivery, the same message can legitimately be delivered and processed more than once, for example if a consumer's deletion request is lost after it already finished processing, or a visibility timeout expires just as processing completes. Because duplicate delivery is a normal, expected outcome of this model rather than a rare edge case, a consumer has to be written so that processing the same message twice produces the same end result as processing it once (an idempotent operation), rather than assuming the queue itself will prevent duplicates.
In the token bucket algorithm, what do the "rate" and "burst" settings each actually control?
The rate is how many tokens get added to the bucket per second, the steady-state throughput the API is designed to sustain indefinitely. The burst is the bucket's total capacity, the maximum number of tokens that can accumulate and therefore the largest spike of concurrent requests the API will accept in a single instant before it starts rejecting anything further. A request consumes one token; as long as the bucket has a token available, the request proceeds immediately, rate and burst are two independent numbers, not one combined setting, because sustained throughput and tolerance for short spikes are genuinely different things to configure.
A client sends a sudden spike of requests that exceeds the configured rate, but the API doesn't immediately reject any of them. Why not, and when does rejection actually start?
The token bucket has accumulated capacity up to the burst limit, if the client had been under the rate limit recently, unused tokens built up in the bucket, and that reserve absorbs a short spike without any request failing, exactly the point of separating burst capacity from steady-state rate. Rejection (a 429 response) only starts once the bucket is actually empty, every accumulated token has been consumed and the spike is sustained long enough that the rate of token consumption keeps exceeding the rate of token replenishment. This is why a token bucket, unlike a hard per-second cap, tolerates brief bursts gracefully while still enforcing a real steady-state ceiling.
Why would a system use a sliding window rate limiter instead of a token bucket, and what problem does it fix?
A naive fixed window (say, "100 requests per minute, resetting on the minute") allows a client to send 100 requests in the last second of one window and another 100 in the first second of the next, 200 requests in a two-second span despite the stated 100/minute limit, an artifact of the window boundary rather than actual demand. A sliding window rate limiter instead evaluates the limit over a continuously moving time range rather than fixed, discrete buckets, which avoids that boundary-doubling effect at the cost of typically more state to track (timestamps of recent requests, not just a single counter) compared to a token bucket's simpler accumulator model.
Why is the CAP theorem often described as "choose two of three," and why is that framing slightly misleading?
The framing suggests a system designer picks any two of consistency, availability, and partition tolerance as a free, standing choice, but partition tolerance isn't actually optional for a real distributed system, network partitions happen (a link fails, a node becomes unreachable), so a system that "chooses" not to tolerate partitions simply isn't distributed in any meaningful sense once one occurs. The real choice CAP describes is narrower and only activates during an actual partition: when nodes can't communicate, do you keep responding and risk inconsistency (AP), or do you pause and refuse some requests to guarantee consistency (CP)? Outside of an actual partition, a well-designed system can be both consistent and available; CAP is a statement about what happens specifically during a partition, not a permanent, constant trade-off.
A banking system typically favors CP behavior during a partition, while a chat application typically favors AP. What does each system actually do differently when a partition occurs, and why does the choice fit each use case?
A CP system, during a partition, pauses or rejects requests that can't be guaranteed consistent, a bank stopping a transfer rather than risking two nodes independently approving withdrawals against the same balance, since a duplicated or lost transaction is a correctness failure worse than a temporary outage. An AP system keeps responding during the partition, accepting the risk of temporarily inconsistent state, a chat app still accepting and displaying messages on both sides of a network split, reconciling them once the partition heals, because staying available and eventually consistent matters more to users than every message reappearing everywhere in a strict, immediate order.
MongoDB can be configured to behave more like a CP system or more like an AP system. What settings make that choice, and what are they actually trading?
Write concern and read preference are the actual levers: a write concern requiring acknowledgment from a majority of replicas favors consistency, a write isn't considered successful until enough nodes agree, at the cost of availability if too many nodes are unreachable during a partition. A looser write concern, or reading from secondaries that might lag behind the primary, favors availability, operations keep succeeding even when full replica agreement isn't achievable, at the cost of potentially reading or acknowledging data that isn't fully consistent across the cluster yet. This is exactly the CP-versus-AP trade-off CAP describes, expressed as a concrete, tunable configuration rather than an abstract theorem.
Kafka vs RabbitMQ
How Kafka and RabbitMQ differ in messaging model, routing, replay, and throughput, and which fits event streaming versus task queues.