Rate limiting is the mechanism that protects a system from being overwhelmed by any single client, whether malicious (abuse, credential-stuffing) or simply buggy (a misconfigured retry loop), this topic covers the four standard algorithms (fixed window, sliding window, token bucket, leaky bucket) with their concrete tradeoffs, why rate limiting must be enforced in shared state rather than per-server memory once a service is horizontally scaled, and the exact HTTP response contract (429, Retry-After) clients rely on.
1. What real flaw does fixed-window rate limiting have?
Interview Questions
2 questions
Cheatsheet
Download-ready reference
Cheatsheet
RATE LIMITING ALGORITHMS:
Fixed window, count per fixed, non-overlapping time bucket.
Simple; FLAW: up to ~2x limit possible in a burst
spanning a window boundary.
Sliding window, continuously-moving window (log or weighted
blend). Fixes boundary-burst flaw; more state.
Token bucket, bucket refills at steady rate up to a capacity;
each request costs 1 token. Allows BURSTS up to
capacity + enforces steady AVERAGE rate. Most
commonly used for client-facing APIs.
Leaky bucket, requests queue and drain at a FIXED constant
rate regardless of input burstiness. Smooths
bursts into steady output; adds queueing latency.
SHARED STATE IS MANDATORY once horizontally scaled: per-server
in-memory counters silently multiply the true effective limit by
the number of instances (each sees only a fraction of a client's
traffic). Fix: Redis (or similar), atomic INCR+EXPIRE or Lua
script, same "externalize state" principle as session storage.
HTTP CONTRACT:
429 Too Many Requests
Retry-After: <seconds> ← real contract, not optional, well-behaved clients back off
for exactly this long.
GRANULARITY: rate limit per-IP (unauthenticated abuse), per-API-
key/user (authenticated abuse), and per-endpoint (cheap health
check vs. expensive search), different risks need different keys.
PERFORMANCE: reject at the EDGE (CDN/gateway) before origin, cheaper
than rejecting after backend capacity is already spent. Token/leaky
bucket need only small fixed per-client state (vs. a full request-
timestamp log for sliding window).
A system with no limit on how much any single client can request is one bad actor (or one buggy retry loop) away from a self-inflicted outage. Rate limiting is the deliberate mechanism for capping how many requests a given client, identified by API key, user ID, or IP address, can make in a given window of time, protecting shared capacity from any one consumer monopolizing it.
FIXED WINDOW: Count requests in fixed, non-overlapping time buckets (e.g. per calendar minute). Limit: 100 requests per minute. Simple to implement (increment a counter, reset it when the window rolls over) but has a real edge-case flaw: a client can send 100 requests in the LAST second of one window and another 100 in the FIRST second of the next window, 200 requests in under 2 seconds, despite a "100/minute" limit, because the two windows don't overlap.SLIDING WINDOW (log or counter): Instead of resetting at fixed boundaries, count requests in a CONTINUOUSLY MOVING window of the last N seconds, recalculated on every request. Fixes fixed window's boundary-burst flaw at the cost of needing to track more state (a log of recent request timestamps, or a weighted blend of the current and previous fixed windows as an approximation).TOKEN BUCKET: A bucket holds up to N tokens, refilling at a steady rate (e.g. +10 tokens/second, capped at 100). Each request consumes 1 token; if the bucket is empty, the request is rejected/delayed. Naturally allows BURSTS up to the bucket's capacity (a client that hasn't made requests in a while has a full bucket and can burst), while still enforcing a steady-state average rate over time, a genuinely useful property fixed/sliding windows don't have.LEAKY BUCKET: Requests enter a queue (the "bucket") and are processed ("leak out") at a FIXED, constant rate, regardless of how bursty the input is. Excess requests beyond queue capacity are dropped. Smooths bursty input into a steady output rate, useful when the downstream system genuinely can't handle ANY burst at all, at the cost of added latency for requests waiting in the queue.
Token bucket is the most commonly used algorithm in practice specifically because "allow reasonable bursts, enforce a steady average" matches how real client traffic actually behaves, a user opening several tabs at once shouldn't be instantly rate-limited the way a sustained flood should be.
Simulate and compare how Token Bucket, Leaky Bucket, Sliding Window, and Fixed Window handle burst traffic and prevent denial of service.
Ingress Traffic: 0 req/sAlgorithm: token bucket
10 / 10 Tokens
200 Allowed0
429 Throttled0
Pass Rate100%
10 requests
3 tokens/sec
0 req/s
Live HTTP Response Feed
Click 'Send Request' or use the Traffic Generator to test
Why rate limiting must live in shared state, not per-server memory#
This is the point where rate limiting directly depends on the scalability topic's core lesson: once a service is horizontally scaled behind a load balancer, a rate limiter that tracks request counts in each server's own local memory is fundamentally broken. If a client's requests get round-robined across 3 servers, each server sees only roughly a third of that client's actual traffic, and each one independently thinks the client is well under the limit, the effective limit the client experiences becomes roughly 3x the intended limit, silently, with no error or warning.
❌ Per-server in-memory rate limiter (3 servers, round robin): Client sends 300 requests, limit is "100/minute": Server A sees ~100 requests → thinks client is exactly at limit Server B sees ~100 requests → thinks client is exactly at limit Server C sees ~100 requests → thinks client is exactly at limit ACTUAL total: 300 requests got through, the true limit was silently tripled by horizontal scaling.✅ Shared-state rate limiter (e.g. Redis, checked by every server): Every server increments/checks the SAME counter for this client, stored in Redis, regardless of which server the request landed on, the count reflects the client's TRUE total across the fleet.
This is exactly the same "externalize state instead of relying on server-local memory" principle from the Scalability topic, applied to a different kind of state (request counts instead of sessions), and it's a genuinely easy trap to fall into, because a naive rate limiter often works correctly in local development (one process, one instance) and only reveals the bug once deployed behind a real load balancer with multiple instances.
When a client exceeds its rate limit, the standard, well-behaved response is HTTP status 429 Too Many Requests, typically paired with a Retry-After header telling the client concretely how long to wait before trying again:
HTTP/1.1 429 Too Many RequestsRetry-After: 30Content-Type: application/json{ "error": "rate_limit_exceeded", "retryAfterSeconds": 30 }
Well-behaved clients (and most modern HTTP client libraries) will honor Retry-After automatically, backing off for the specified duration before retrying, this is a real, load-bearing part of the contract, not just a nicety, because a client that retries immediately on a 429 without honoring Retry-After just adds to the load the rate limiter was trying to protect against in the first place.
Your API uses fixed-window rate limiting at 100 requests/minute, with windows aligned to the top of each minute (:00 to :59). A client sends 100 requests at 12:00:59 and another 100 requests at 12:01:00. Does this violate the "100 requests per minute" limit, and why or why not according to the algorithm as implemented?
Solution
According to the fixed-window algorithm as implemented, this does NOT violate the limit, even though it's clearly 200 requests within roughly 1 second of wall-clock time. The first 100 requests fall in the window 12:00:00,12:00:59, and the second 100 fall in the next window, 12:01:00,12:01:59. Each window independently sees exactly 100 requests, which is at (not over) the limit, the algorithm has no memory of the previous window when evaluating the current one.
This is precisely the boundary-burst flaw fixed window has: it's technically compliant with "100 per minute, evaluated per fixed window" while being wildly non-compliant with the intent behind that limit (roughly even request pacing, no massive bursts). A sliding window algorithm would correctly catch this, because it evaluates the last 60 seconds continuously rather than in fixed, non-overlapping buckets, from a sliding window's perspective, there were 200 requests within the last 60-second span at 12:01:00, correctly triggering the limit.
This is genuinely how production token-bucket limiters work, just backed by shared state (Redis, typically using INCR plus EXPIRE, or a Lua script for atomicity) instead of a single in-process object, the refill-based-on-elapsed-time math shown here is the real core logic, not a simplification of it.
A shared, atomic counter check under concurrent access, the exact correctness requirement a Redis-backed rate limiter has to satisfy, is the same class of problem as coordinating concurrent access to a single-threaded event queue, covered in The Node.js Event Loop: the rate limiter's atomicity requirement exists precisely because, without it, concurrent requests can interleave in ways a single-threaded mental model wouldn't predict.
1. Implementing rate limiting with per-process in-memory state behind a load balancer#
// ❌ Works fine locally (1 process); silently multiplies the effective// limit by the number of server instances in productionconst requestCounts = new Map(); // lives only in THIS process's memoryfunction isAllowed(clientId) { const count = requestCounts.get(clientId) ?? 0; if (count >= 100) return false; requestCounts.set(clientId, count + 1); return true;}
As covered in Concept, this silently multiplies the true effective limit by the number of server instances, since each instance only sees a fraction of a given client's total traffic. The fix is tracking counts in shared state (Redis or similar) that every instance reads from and writes to.
2. Returning a bare 429 with no Retry-After header#
HTTP/1.1 429 Too Many Requests// ❌ no Retry-After, client has no idea how long to wait, and may// retry immediately, adding more load to an already-overloaded limiter
Without Retry-After, well-behaved clients are left guessing how long to back off, and poorly-behaved ones may retry immediately, defeating much of the point of rate limiting in the first place. Always pair a 429 with a concrete Retry-After value.
3. Using fixed-window limiting where burst-at-the-boundary behavior actually matters#
"Our limit is 1000 requests/minute, fixed window, aligned to the clock." // ❌ if bursts near window boundaries are a real concern
As shown in Try It, fixed window allows up to 2x the stated limit in a worst-case boundary burst. If genuinely even pacing matters (e.g. protecting a fragile downstream system), sliding window or token/leaky bucket are more accurate choices, fixed window's simplicity comes at the cost of this specific, well-known gap.
Enforce rate limits in shared state (Redis or equivalent) reachable by every server instance, never per-process memory, once horizontally scaled.
Prefer token bucket for client-facing APIs where occasional legitimate bursts (a user opening multiple tabs, a client retrying after a network blip) shouldn't be punished the same as sustained abuse.
Always return 429 with a Retry-After header, this is a real contract well-behaved clients depend on, not an optional nicety.
Rate limit at multiple granularities where appropriate, per-IP (catches unauthenticated abuse), per-API-key/user (catches abuse from an authenticated but misbehaving client), and sometimes per-endpoint (a cheap health-check endpoint can tolerate a much higher limit than an expensive search endpoint).
Enforce rate limits as close to the edge as practical (CDN/API gateway before origin servers), rejecting an over-limit request before it consumes real backend capacity is strictly cheaper than rejecting it deeper in the stack.
Token bucket and leaky bucket both require only a small, fixed amount of state per client (a token count and a timestamp, or a queue depth), this is cheap to store and check even at very high request volumes, unlike a sliding-window log approach that tracks every individual request timestamp.
Checking and decrementing a rate limit counter in Redis should be done atomically (e.g. via a Lua script or Redis's INCR+EXPIRE combination), a naive read-then-write from application code is a race condition under concurrent requests, letting more requests through than the limit intends.
Rejecting requests early (at the edge, before touching a database or doing expensive computation) means the cost of an over-limit request is just the rate-limit check itself, nearly free compared to letting it partially execute before being rejected downstream.
Math.
min
(
this
.capacity,
this
.tokens
+
tokensToAdd);
this.lastRefill = now;
}
tryConsume(tokens = 1) {
this._refill(); // top up based on elapsed time since last check
if (this.tokens >= tokens) {
this.tokens -= tokens;
return true; // request allowed
}
return false; // request rejected, not enough tokens
}
}
// Allow bursts up to 10 requests, refilling at 2 tokens/second
// (steady-state average = 2 requests/second, but bursts up to 10 are fine)
const limiter = new TokenBucket({ capacity: 10, refillRatePerSecond: 2 });
// A burst of 10 rapid requests: all succeed (bucket starts full)
