Search
7 results for “consistent-hashing”
Search results
Consistent Hashing
Why placing hosts and keys on a hash ring means adding or removing one host out of N only remaps roughly 1/N of the keys, instead of the near-total remapping a plain modulo hash would force on every single change.
Why does a naive `hash(key) % N` scheme for distributing keys across N servers fall apart the moment a server is added or removed?
With plain modulo hashing, the server a key maps to depends directly on the current value of N, since almost every key's `hash(key) % N` result changes the instant N changes to N-1 or N+1, even though the underlying hash values themselves didn't change at all. That means adding or removing a single server can remap the overwhelming majority of keys to different servers simultaneously, which for a cache means a massive wave of cache misses, and for a sharded store means a massive, unnecessary data-migration event, triggered by a change to just one server out of many.
How does consistent hashing (a ring hash) avoid that near-total remapping problem?
Instead of computing `hash(key) % N`, both hosts and keys are hashed onto positions on a fixed conceptual ring (typically the hash function's full output range), and each key is assigned to the next host found by walking clockwise from the key's position. Removing a host only affects the keys that were mapped to that specific host's section of the ring, they get reassigned to the next host further along, while every other key on the ring, owned by a different host's section entirely, is completely unaffected. For a ring hash across N hosts, adding or removing one host affects only about 1/N of the total keys, not nearly all of them.
Why does a real ring hash implementation give each host many positions on the ring instead of just one, and what problem would a single position per host cause?
A host thrown onto the ring at just one point can end up, purely by chance, owning a disproportionately large or small arc of the ring if the hash values happen to land unevenly, since with few points there's no averaging effect smoothing out the randomness. Assigning each host many positions on the ring, scaled by that host's intended weight, so a double-weight host gets roughly twice as many ring entries as a single-weight one, averages out that randomness across many smaller arcs per host, producing a much more even overall traffic distribution than a single coin-flip-like placement per host would.
A custom class defines __eq__ based on a value field but leaves __hash__ using default identity-based hashing. What actually goes wrong when you use an instance as a dict key?
In Python 3, simply defining `__eq__` without touching `__hash__` doesn't leave the old identity-based hash in place, Python automatically sets `__hash__` to `None` on that class, making instances unhashable, so using one as a dict key raises `TypeError` immediately rather than corrupting anything. This happens even if a parent class defines `__hash__`: overriding `__eq__` in a subclass sets that subclass's `__hash__` to `None` regardless of what the parent provides, ordinary inheritance does not carry the parent's hash forward. The silent, no-exception version of this bug only happens if the class explicitly keeps or re-supplies an identity-based `__hash__` alongside the value-based `__eq__` (e.g. `__hash__ = object.__hash__`, or, to retain a parent's hash on purpose, `__hash__ = Parent.__hash__`). In that case, two instances with the same value compare equal (`a == b` is `True`) but hash differently, and inserting under key `a` then looking up with an equal-but-distinct key `b` lands in the wrong hash bucket and returns nothing, because the hash mismatch never gave the lookup a chance to even check equality against the right entry.
Python's own documentation warns that raising the recursion limit "should be done with care, because a too-high limit can lead to a crash." Why doesn't raising the limit simply allow deeper, safe recursion?
The recursion limit is a proxy for the real constraint, actual available C stack space, which is platform-dependent and finite regardless of what the configured limit says. Setting the limit higher than the platform's actual available stack can support doesn't create more stack space, it just removes the early warning that would have raised a clean `RecursionError`, so recursion can now run deep enough to exhaust the real stack and crash the process with a low-level segmentation fault instead, a worse failure mode than the exception the limit was preventing in the first place.
PostgreSQL's documentation says it's "impossible to suppress nested-loop joins entirely" even with enable_nestloop off. What does that tell you about relying on planner hints to force a specific join algorithm?
That specific guarantee is documented only for `enable_nestloop`: turning it off only discourages the planner by making nested loops look artificially expensive in cost estimation, it can't hard-disable them, because for some queries a nested loop is the only viable plan at all (for example, certain correlated subquery shapes), so the planner will still use one if it must. `enable_hashjoin` and `enable_mergejoin` don't carry that same caveat, disabling either one actually can prevent the planner from choosing that join type, since a nested loop (or the other remaining method) is always available as a fallback plan. In practice, though, all three settings are best treated as a debugging/diagnostic tool for understanding planner behavior, not a reliable production mechanism for forcing a specific join algorithm, the actual fix for a bad plan is almost always better statistics (via `ANALYZE`) or a better index, not overriding the planner's method choice.