Rate limiting tends to arrive late in a project’s life, usually the day after an automated script hammers an endpoint, a bill spikes, or a single misbehaving client takes the whole service down for everyone else. Bolted on in a panic, it becomes a crude gate that frustrates legitimate users as much as it stops abusers. Treated as a deliberate design decision from the start, it becomes something better: a way to protect your service, share capacity fairly, and communicate clearly with the clients that depend on you.
What You Are Actually Protecting
Before choosing an algorithm, it helps to be honest about what rate limiting is for, because different goals lead to different designs. The reasons usually fall into a few buckets.
- Stability, so that no single caller can consume enough resources to degrade the experience for everyone else.
- Cost control, since every request may translate into database load, third-party API charges, or compute you have to pay for.
- Fairness, giving each customer a reasonable slice of shared capacity rather than letting the loudest one dominate.
- Abuse prevention, slowing down credential-stuffing attempts, scraping, and brute-force attacks that depend on high request volume.
These goals are not identical. A limit designed to stop a brute-force login attack should be tight and tied to an account or IP, while a limit meant to smooth out load can be generous and forgiving of short bursts. Knowing which problem you are solving keeps you from applying one blunt rule everywhere.
The Common Algorithms and Their Trade-offs
There are a handful of standard approaches, and each has a distinct personality. A fixed window counter allows a set number of requests per calendar interval, such as one hundred per minute, and resets on the minute boundary. It is simple and cheap, but it has an unfair edge: a client can send one hundred requests in the last second of one window and another hundred in the first second of the next, effectively doubling the intended rate across the boundary.
A sliding window smooths this out by considering a rolling period rather than a fixed boundary, so the limit feels consistent no matter when the requests arrive. A token bucket takes a different and often more pleasant approach: tokens are added to a bucket at a steady rate, each request spends one token, and the bucket has a maximum size. This naturally allows short bursts up to the bucket’s capacity while still enforcing a sustainable average rate, which matches how real clients behave. A leaky bucket, by contrast, processes requests at a strictly constant rate and is useful when downstream systems cannot tolerate bursts at all. For most public APIs, the token bucket is a good default because it is both fair and tolerant of the natural burstiness of real traffic.
Choose the Right Key to Limit On
A limit is only as good as the thing it counts against. Limiting purely by IP address is easy but blunt, because many legitimate users can share one address behind a corporate network or mobile carrier, while a determined attacker can rotate through many addresses. Limiting by authenticated account is usually fairer for a logged-in API, since it ties usage to the entity actually responsible for it. Often the right answer is a layered approach: a broad per-IP limit to catch anonymous floods, plus a per-account limit for authenticated traffic, plus tighter per-endpoint limits on expensive or sensitive operations such as login, search, or file upload.
Different customers may also deserve different limits. A free tier and a paid tier can share the same code path while carrying different quotas, which turns rate limiting from a purely defensive mechanism into part of your product’s structure rather than an obstacle grafted onto it.
Say No Politely and Clearly
How you reject a request matters as much as when you reject it. The agreed convention is to respond with the HTTP status 429, meaning too many requests, rather than a generic error that clients cannot distinguish from a real failure. Alongside it, a Retry-After header tells the client exactly how long to wait before trying again, which removes the guesswork that leads to aggressive retry loops.
It is also good practice to include headers that expose the current state of the limit: how many requests are allowed in the window, how many remain, and when the window resets. This lets a well-behaved client pace itself proactively instead of blindly sending requests until it hits the wall. A rate limit that communicates its own rules is far easier to integrate against, and it dramatically reduces the support burden of developers asking why their requests are being rejected.
Help Clients Behave Well
Rate limiting is a two-sided relationship, and the client half is often neglected. A client that receives a 429 and immediately retries in a tight loop only makes the congestion worse. The correct pattern is exponential backoff with jitter: wait a short interval before the first retry, roughly double the wait after each subsequent failure, and add a small random component so that many clients recovering at once do not synchronize into repeated waves. Respecting the Retry-After value when the server provides it is even better than guessing.
If you both operate the server and publish the client library, building this behavior into the library means every integrator gets it for free and your service stays healthier under stress.
The Hard Part: Doing It Across Many Servers
Rate limiting is straightforward on a single machine and genuinely tricky across a fleet. If your service runs on ten servers and each keeps its own local counter, a client can quietly get ten times the intended limit by spreading requests across them. The usual solution is a shared, fast data store such as an in-memory cache that all servers consult, so the counter is global rather than per-instance. This introduces its own considerations around latency and atomic updates, but it is the price of enforcing a coherent limit at scale.
The broader lesson is that rate limiting is a design surface, not a switch. Decide early what you are protecting, pick an algorithm that matches how your clients actually behave, key it on something meaningful, and communicate your decisions through clear responses. Done this way, it stops being a punishment you inflict on users and becomes part of what makes your service dependable.