🚦 Rate Limiting and Throttling

Why Rate Limiting Matters

Rate limiting caps how many requests a client may make in a given time window. It sounds defensive, and it is — but it also protects the honest majority of your users from the noisy few.

💡 A useful analogy: Picture a nightclub with a fixed capacity. The bouncer only lets so many people in per minute. Once it's full, newcomers wait. Nobody's night is ruined by overcrowding, and the fire marshal stays happy. A rate limiter is that bouncer for your API.

Without a limiter, an API is exposed to several failure modes at once:

• Denial of service — a flood of requests exhausts CPU, memory, or database connections.
• The "noisy neighbor" — one heavy client degrades latency for everyone sharing the service.
• Runaway cost — usage-billed downstreams (databases, third-party APIs, LLMs) can generate a shocking invoice.
• Brute-force attacks — unlimited login attempts make credential-guessing trivial.
• Cascading failure — an overloaded service takes its dependencies down with it.

Rate limiting turns each of these from a catastrophe into a polite 429 Too Many Requests.

Limiting, Throttling & Quotas

These three terms overlap in casual speech but describe distinct behaviors. Knowing the difference helps you design the right policy.

Most production APIs combine all three: a short-window rate limit to smooth traffic, throttling to shape bursts, and a monthly quota tied to a pricing plan. For example, a plan might allow 100 requests/minute (rate limit) up to 100,000 requests/month (quota).

The Four Core Algorithms

Every rate limiter is one of a handful of algorithms. Each trades off simplicity, memory, and how it handles bursts.

1. Fixed Window Counter

Divide time into fixed blocks (say, one minute) and count requests in the current block. Reset the counter when the block rolls over. Dead simple, but it has a famous flaw: a client can send a full window's worth of requests at 10:00:59 and another full window at 10:01:00 — double the intended rate across that boundary. This is the edge-spike problem.

2. Sliding Window Counter

Smooths the edge spike by blending the current and previous windows with a weight based on how far into the current window you are. Nearly as cheap as fixed window, but far more even. A great default for most APIs.

3. Token Bucket

A bucket holds up to N tokens and refills at a steady rate. Each request spends a token; if the bucket is empty, the request is rejected or delayed. The bucket's capacity allows short bursts (spend saved-up tokens) while the refill rate enforces the long-term average. This flexibility makes it the most widely used algorithm — it's what AWS API Gateway and Stripe use.

4. Leaky Bucket

Requests enter a queue and are processed ("leak out") at a fixed rate. Overflow is dropped. Unlike the token bucket, it smooths output completely — no bursts pass through — which is ideal when a downstream system needs a perfectly steady feed.

Building a Token Bucket

The token bucket is worth implementing yourself once — it clarifies the whole idea. The key trick is lazy refill: instead of a timer topping up tokens, we calculate how many tokens should have accrued based on elapsed time whenever a request arrives.

class TokenBucket {
  constructor(capacity, refillPerSecond) {
    this.capacity = capacity;         // max burst size
    this.refillRate = refillPerSecond; // sustained rate
    this.tokens = capacity;           // start full
    this.lastRefill = Date.now();
  }

  #refill() {
    const now = Date.now();
    const elapsed = (now - this.lastRefill) / 1000;
    // Add accrued tokens, capped at capacity
    this.tokens = Math.min(this.capacity, this.tokens + elapsed * this.refillRate);
    this.lastRefill = now;
  }

  tryConsume(cost = 1) {
    this.#refill();
    if (this.tokens >= cost) {
      this.tokens -= cost;
      return true;   // allowed
    }
    return false;    // rate limited
  }

  // Seconds until enough tokens exist (for a Retry-After header)
  retryAfter(cost = 1) {
    this.#refill();
    if (this.tokens >= cost) return 0;
    return (cost - this.tokens) / this.refillRate;
  }
}

Wiring it into Express, keyed per client, gives a working limiter:

const buckets = new Map(); // key -> TokenBucket

function rateLimit(req, res, next) {
  const key = req.user?.id ?? req.ip;
  if (!buckets.has(key)) {
    buckets.set(key, new TokenBucket(10, 1)); // burst 10, 1 req/sec
  }
  const bucket = buckets.get(key);

  if (bucket.tryConsume()) return next();

  const wait = Math.ceil(bucket.retryAfter());
  res.set('Retry-After', String(wait));
  res.status(429).json({ error: 'Rate limit exceeded', retryAfter: wait });
}

app.use('/api/', rateLimit);

Where & How to Enforce

Rate limiting can live at several layers, and the right choice depends on how much traffic you want to stop before it costs you resources.

• Edge / gateway (Nginx, Cloudflare, AWS API Gateway) — cheapest place to shed abusive traffic, before it hits your app at all.
• Application (Express/Django/Laravel middleware) — where you have the richest context (which user, which endpoint) for precise limits.

You also choose the granularity — the key you count against:

A common pattern is stricter limits on sensitive routes — for instance, five login attempts per hour but a hundred general requests per minute:

import rateLimit from 'express-rate-limit';

const apiLimiter = rateLimit({
  windowMs: 60_000, max: 100,            // 100/min for the API
  standardHeaders: true, legacyHeaders: false,
});

const authLimiter = rateLimit({
  windowMs: 60 * 60_000, max: 5,         // 5/hour for login
  message: 'Too many login attempts, try again later',
});

app.use('/api/', apiLimiter);
app.use('/api/auth/login', authLimiter); // stacks a tighter limit

Distributed Limiting with Redis

When your API runs on multiple servers, the limit must be shared state. Redis is the standard choice: it's fast, centralized, and supports atomic operations so concurrent requests can't slip through a race condition.

The classic fixed-window counter in Redis uses INCR plus EXPIRE. To make the two operations atomic, run them as a single Lua script so no request is counted without also setting the expiry:

import Redis from 'ioredis';
const redis = new Redis(process.env.REDIS_URL);

// Atomic: increment the window counter and set TTL on first hit
const script = `
  local current = redis.call('INCR', KEYS[1])
  if current == 1 then
    redis.call('EXPIRE', KEYS[1], ARGV[1])
  end
  return current
`;

async function checkLimit(id, limit, windowSec) {
  const bucket = Math.floor(Date.now() / (windowSec * 1000));
  const key = `rl:${id}:${bucket}`;
  const count = await redis.eval(script, 1, key, windowSec);

  return {
    allowed: count <= limit,
    limit,
    remaining: Math.max(0, limit - count),
    reset: (bucket + 1) * windowSec, // unix seconds when window resets
  };
}

Using it as middleware, every server instance now shares one authoritative counter:

app.use('/api/', async (req, res, next) => {
  const id = req.user?.id ?? req.ip;
  const { allowed, limit, remaining, reset } = await checkLimit(id, 100, 60);

  res.set({
    'RateLimit-Limit': limit,
    'RateLimit-Remaining': remaining,
    'RateLimit-Reset': reset,
  });

  if (allowed) return next();
  res.set('Retry-After', reset - Math.floor(Date.now() / 1000));
  res.status(429).json({ error: 'Rate limit exceeded' });
});

The 429 Response Contract

How you reject matters as much as the limit itself, because well-behaved clients rely on your response to back off. The conventions are simple and standardized.

• Return 429 Too Many Requests — not 403, not 503.
• Include a Retry-After header (seconds, or an HTTP date) telling the client when to try again.
• Advertise the policy with RateLimit-Limit, RateLimit-Remaining, and RateLimit-Reset on every response, not just rejections.
• Give a clear, machine-readable error body and a docs link.

HTTP/1.1 429 Too Many Requests
Content-Type: application/json
RateLimit-Limit: 100
RateLimit-Remaining: 0
RateLimit-Reset: 1618884000
Retry-After: 42

{
  "error": {
    "code": "rate_limit_exceeded",
    "message": "You have exceeded 100 requests per minute.",
    "retryAfter": 42,
    "docs": "https://api.example.com/docs/rate-limits"
  }
}

On the client side, honor Retry-After and add exponential backoff with jitter so that a fleet of clients doesn't all retry in the same instant (a "thundering herd"):

async function fetchWithRetry(url, opts = {}, attempt = 0) {
  const res = await fetch(url, opts);
  if (res.status !== 429 || attempt >= 5) return res;

  const retryAfter = Number(res.headers.get('Retry-After')) || 2 ** attempt;
  const jitter = Math.random() * 0.3 * retryAfter;  // spread out retries
  await new Promise(r => setTimeout(r, (retryAfter + jitter) * 1000));
  return fetchWithRetry(url, opts, attempt + 1);
}

Hands-on: Bucket in Action

const bucket = new TokenBucket(5, 1); // capacity 5, 1/sec

const burst = [];
for (let i = 0; i < 7; i++) burst.push(bucket.tryConsume());
console.log(burst);
// [true, true, true, true, true, false, false]
//  first 5 succeed (capacity), last 2 rejected (empty)

await new Promise(r => setTimeout(r, 2000)); // ~2 tokens refill

console.log(bucket.tryConsume(), bucket.tryConsume());
// true true  — the bucket recovered at 1 token/second

Best Practices

Summary & Quiz

📚 Further Reading

• RFC 6585 — 429 Too Many Requests
• express-rate-limit (npm)
• Stripe engineering — Scaling your API with rate limiters

🚀 What's Next?

Rate limiting reduces how often you have to do work. The next lesson reduces how often you have to redo it: API response caching strategies — storing responses so repeated requests are answered in milliseconds.