Search
6 results for “rate-limiting”
Search results
Rate Limiting Algorithms
How the token bucket algorithm AWS API Gateway actually uses separates a steady-state rate from a burst allowance, and why a request only fails once the bucket is genuinely empty, not the instant the average rate is exceeded.
In the token bucket algorithm, what do the "rate" and "burst" settings each actually control?
The rate is how many tokens get added to the bucket per second, the steady-state throughput the API is designed to sustain indefinitely. The burst is the bucket's total capacity, the maximum number of tokens that can accumulate and therefore the largest spike of concurrent requests the API will accept in a single instant before it starts rejecting anything further. A request consumes one token; as long as the bucket has a token available, the request proceeds immediately, rate and burst are two independent numbers, not one combined setting, because sustained throughput and tolerance for short spikes are genuinely different things to configure.
A client sends a sudden spike of requests that exceeds the configured rate, but the API doesn't immediately reject any of them. Why not, and when does rejection actually start?
The token bucket has accumulated capacity up to the burst limit, if the client had been under the rate limit recently, unused tokens built up in the bucket, and that reserve absorbs a short spike without any request failing, exactly the point of separating burst capacity from steady-state rate. Rejection (a 429 response) only starts once the bucket is actually empty, every accumulated token has been consumed and the spike is sustained long enough that the rate of token consumption keeps exceeding the rate of token replenishment. This is why a token bucket, unlike a hard per-second cap, tolerates brief bursts gracefully while still enforcing a real steady-state ceiling.
Why would a system use a sliding window rate limiter instead of a token bucket, and what problem does it fix?
A naive fixed window (say, "100 requests per minute, resetting on the minute") allows a client to send 100 requests in the last second of one window and another 100 in the first second of the next, 200 requests in a two-second span despite the stated 100/minute limit, an artifact of the window boundary rather than actual demand. A sliding window rate limiter instead evaluates the limit over a continuously moving time range rather than fixed, discrete buckets, which avoids that boundary-doubling effect at the cost of typically more state to track (timestamps of recent requests, not just a single counter) compared to a token bucket's simpler accumulator model.
Python's own documentation warns that raising the recursion limit "should be done with care, because a too-high limit can lead to a crash." Why doesn't raising the limit simply allow deeper, safe recursion?
The recursion limit is a proxy for the real constraint, actual available C stack space, which is platform-dependent and finite regardless of what the configured limit says. Setting the limit higher than the platform's actual available stack can support doesn't create more stack space, it just removes the early warning that would have raised a clean `RecursionError`, so recursion can now run deep enough to exhaust the real stack and crash the process with a low-level segmentation fault instead, a worse failure mode than the exception the limit was preventing in the first place.
A query filters WHERE logdate >= 2008-01-01 against a table range-partitioned by logdate across dozens of monthly partitions. What does partition pruning actually do, and what does it depend on?
Partition pruning lets the planner prove, from the query's WHERE clause and each partition's declared bounds, that some partitions cannot possibly contain a matching row, and it excludes them from the plan entirely rather than scanning and filtering every partition. In this example, `logdate >= 2008-01-01` has no upper bound, so it prunes only the partitions entirely before 2008-01, decades of older monthly partitions are eliminated before execution, while every partition from 2008-01 onward, including all of them up to the present, is still considered and scanned. Pruning down to a single partition would need a bounded predicate on both ends, for example `logdate >= 2008-01-01 AND logdate < 2008-02-01`. This depends entirely on the partition bounds themselves, not on any index, a partitioned table with no indexes at all still benefits from pruning, and pruning specifically requires the WHERE clause to reference the partition key directly with values (or parameters) the planner can actually compare against those bounds.