Skip to main content

Please Slow Down

API rate limits, explained for junior developers and non-dev builders

· 18 min read
Samuel Rossille
Samuel Rossille
Chief Technology Officer
Cover

Integrating a third-party API is the kind of thing that is easy to get working, but hides a lot of subtleties in the corners. Rate limiting is one of them. Almost everyone learns it the hard way, and in the same order: first you don't know rate limits exist, then you add a retry, then you discover why that retry was a bad idea, and so on, one lesson at a time, each fix revealing the next problem.

This post walks that path, step by step. It is written for junior developers, and for the tech-savvy people who sometimes end up "just connecting" an API to something. At the end of it, you will know what a 429 is, how to behave like a good citizen with someone else's API, and the one situation where paying for a higher limit is the right call.

  1. 0 UnawareFire everything
  2. 1 RetryTry again at once
  3. 2 Back offWait longer each time
  4. 3 JitterWait a random time
  5. 4 PaceStay under the limit
  6. 5 ShareOne budget for all
  7. 6 SaveOnly useful calls
  8. 7 PayBuy a higher limit
Eight steps, from "what is a 429?" to "we should pay for more".

Throughout the post, we use one example. You need to enrich companies in your CRM with data from a company data API: legal name, address, headcount, activity. You send a registration number, you get the company back. The API's documentation says you can make 10 requests per second. To keep the charts readable, we enrich 200 companies. In real life it would be 50,000, and every problem below would be 250 times bigger.

Step 0: you don't know rate limits exist​

  1. 0 Unaware
  2. 1 Retry
  3. 2 Back off
  4. 3 Jitter
  5. 4 Pace
  6. 5 Share
  7. 6 Save
  8. 7 Pay

The first version is the obvious one. Get the list, fetch every company, save the result.

const companies = await crm.listCompaniesWithMissingData()

await Promise.all(
companies.map(async (company) => {
const response = await fetch(`${COMPANY_API_URL}/companies/${company.registrationNumber}`, {
headers: {Authorization: `Bearer ${COMPANY_API_KEY}`},
})
if (!response.ok) {
logger.error(`Could not enrich company ${company.id}: HTTP ${response.status}`)
return
}
await crm.update(company.id, await response.json())
}),
)

It runs in a fraction of a second. And the logs are full of errors.

0102030limit: 10 per secondoff the chart: up to 200 calls in a single second0s10s20s30s40s50s60s
200 OK429 Too Many Requests
10 / 200
companies enriched
190
failed with a 429
190
calls rejected with a 429
All 200 requests leave at once. The API accepts 10. The 190 others end up as errors in the logs.

Promise.all sent 200 requests at the same instant. The API answered 10 of them. The 190 others got this:

HTTP/1.1 429 Too Many Requests
Content-Type: application/json
Retry-After: 1
RateLimit-Policy: "default";q=10;w=1
RateLimit: "default";r=0;t=1

{"error": "rate_limited", "message": "Slow down. Your plan allows 10 requests per second."}

What a 429 says​

429 Too Many Requests is the HTTP status code for "you are asking too often". It is not a bug on their side, and it is not a permanent failure. It means not now. In most APIs the request was not processed at all, so trying again later is safe.

The response usually tells you how much later:

  • Retry-After is the standard header. Either a number of seconds (Retry-After: 1) or a date.
  • Quota headers tell you where you stand. Names vary from one provider to the next (X-RateLimit-Remaining, X-RateLimit-Reset, and so on). An IETF draft is converging on RateLimit and RateLimit-Policy, like in the example above: a quota of 10 per 1-second window, 0 remaining, reset in 1 second.
  • The body often says which limit you hit. Read it once, it is cheaper than guessing.

Why APIs do this​

The provider has a finite number of servers, shared by all its customers. Without limits, one runaway script slows down everyone else. So each API key gets a budget, and the most common way to enforce it is a token bucket.

+10 tokens per secondholds 10 maxThe API key's budgeteach requesttakes 1 tokenToken available200 OK, here is your dataBucket empty429 Too Many Requests, come back later
A token bucket. Bursts are fine while there are tokens left. After that, you get exactly the refill rate.

Every request takes a token. Tokens come back at a fixed rate. When the bucket is empty, you get a 429. This explains a behavior that puzzles everybody the first time: a short burst works fine (there were tokens in the bucket), and then everything fails at once (there weren't anymore).

Limits come in several flavors, and one API often has a few of them at the same time:

  • Per second or per minute: the bucket above.
  • Per day or per month: a quota tied to your plan. Hitting it means waiting hours, not seconds.
  • Concurrency: no more than N requests in flight at once, however slow they are.
  • Per endpoint: a search endpoint is often far stricter than a "get by id" one.

The documentation lists them. From now on, you read that page before you write the loop.

Step 1: "requests can fail, let's retry"​

  1. 0 Unaware
  2. 1 Retry
  3. 2 Back off
  4. 3 Jitter
  5. 4 Pace
  6. 5 Share
  7. 6 Save
  8. 7 Pay

The natural reaction to step 0 is: some requests fail, so retry them until they work.

async function fetchCompany(registrationNumber: string): Promise<Response> {
while (true) {
const response = await fetch(`${COMPANY_API_URL}/companies/${registrationNumber}`, {headers})
if (response.status !== 429) return response
}
}

And it works! Every company gets enriched, in about 20 seconds. Look at the red part, though.

0102030limit: 10 per secondoff the chart: up to 1,910 calls in a single second0s10s20s30s40s50s60s
200 OK429 Too Many Requests
200 / 200
companies enriched
19 s
until the last one is done
19,000
calls rejected with a 429
Each rejected request bounces back immediately, gets rejected again, and bounces back. The bars are cut at 30: the real peak is almost 2,000 calls in one second.

Retrying immediately turned 200 jobs into almost 20,000 requests. And it bought nothing: the script finishes in 20 seconds because 200 companies at 10 per second take 20 seconds. The other 19,000 calls were pure noise.

In real life, it is worse than the simulation:

  • Many APIs count rejected requests against your quota. The retries eat the very tokens they are waiting for, and the script never finishes.
  • Some APIs block keys that hammer them. For an hour, or until you email support.
  • It does not scale down. At 50,000 companies, this loop is a small denial-of-service attack with your company's name on it.
A 429 is the API asking you to slow down. Retrying immediately is answering "no".

Step 2: wait before retrying​

  1. 0 Unaware
  2. 1 Retry
  3. 2 Back off
  4. 3 Jitter
  5. 4 Pace
  6. 5 Share
  7. 6 Save
  8. 7 Pay

If the API says "too many", the fix is to send fewer. So after a 429, you wait. And if you get another 429, you wait longer: 1 second, then 2, then 4, then 8, up to a cap. That's exponential backoff. If the API is overloaded, you give it room to breathe instead of piling on.

for (let attempt = 0; ; attempt++) {
const response = await fetch(url, {headers})
if (response.status !== 429) return response
await sleep(Math.min(MAX_DELAY_MS, 1_000 * 2 ** attempt))
}

Ten times fewer rejected calls. But look at the total time.

0102030limit: 10 per secondoff the chart: up to 200 calls in a single second0s10s20s30s40s50s60s
200 OK429 Too Many Requests
200 / 200
companies enriched
7 min 59 s
until the last one is done
1,900
calls rejected with a 429
A neat burst of 10 successes, then silence, then another burst. The waves get further and further apart.

Twenty seconds became eight minutes. The script is now perfectly polite and terribly slow. Here is why. All 190 rejected requests were rejected at the same moment. They all wait exactly 1 second. So they all come back at the same moment, 10 get in, and the 180 others all wait exactly 2 seconds, and come back together again.

Retry every secondall together, foreverExponential backoff1, 2, 4, 8 seconds…0s4s8s12s16s
Eight clients rejected at the same moment. Each dot is a retry. Vertical lines of dots are waves hitting the API together.

This is called a thundering herd. Backoff fixed the volume of retries, but not their timing. Every wave hits an API that can only take 10, and everything else waits for the next wave, which keeps getting further away.

Step 3: wait a random time​

  1. 0 Unaware
  2. 1 Retry
  3. 2 Back off
  4. 3 Jitter
  5. 4 Pace
  6. 5 Share
  7. 6 Save
  8. 7 Pay

The fix for the herd is jitter: instead of waiting exactly 4 seconds, each request waits a random time between 0 and 4 seconds. Same average patience, but the requests spread out and fill the free slots one by one.

Exponential backoff1, 2, 4, 8 seconds…Backoff + jitterrandom waits0s4s8s12s16s
Same eight clients. Without jitter, they move as a herd. With jitter, they spread over time.
0102030limit: 10 per secondoff the chart: up to 409 calls in a single second0s10s20s30s40s50s60s
200 OK429 Too Many Requests
200 / 200
companies enriched
43 s
until the last one is done
789
calls rejected with a 429
The retries spread out and fill the free slots. Red for the first seconds, then a steady flow.

From eight minutes down to about 40 seconds. This is the right way to retry, and turning it into a reusable helper is where a few more details show up:

const BASE_DELAY_MS = 1_000
const MAX_DELAY_MS = 32_000
const MAX_ATTEMPTS = 8

async function callWithBackoff(request: () => Promise<Response>): Promise<Response> {
for (let attempt = 0; ; attempt++) {
const response = await request()
if (!isRetryable(response.status) || attempt === MAX_ATTEMPTS) return response

// "Full jitter": a random wait between 0 and the exponential window
const window = Math.min(MAX_DELAY_MS, BASE_DELAY_MS * 2 ** attempt)
const jittered = Math.random() * window
// If the server said when to come back, never come back earlier
const serverAsked = retryAfterMs(response.headers.get('retry-after')) ?? 0
await sleep(Math.max(jittered, serverAsked))
}
}

function isRetryable(status: number): boolean {
return status === 429 || status === 502 || status === 503 || status === 504
}

function retryAfterMs(header: string | null): number | undefined {
if (!header) return undefined
const seconds = Number(header)
if (!Number.isNaN(seconds)) return seconds * 1_000
const date = Date.parse(header)
return Number.isNaN(date) ? undefined : Math.max(0, date - Date.now())
}

Each line of that helper answers a question you run into the first time you write it:

  • The server knows better than you. When Retry-After is there, it wins over your own computation.
  • Some errors never heal. A 429 or a 503 will pass later. A 400 or a 404 will fail the same way forever, so retrying them only burns budget.
  • Retries must end. After a maximum number of attempts, the job fails and is reported like any other error, so a human can look at it.
  • Retrying a write can do it twice. A 429 usually means "not processed". A timeout means "maybe processed". When an API supports idempotency keys, send one with every write, or one day you will create the same invoice twice.

Step 3 is where most integrations stop. But look at the chart again: almost 800 requests were still rejected. The script still learns the limit by hitting it, again and again.

Step 4: stay under the limit​

  1. 0 Unaware
  2. 1 Retry
  3. 2 Back off
  4. 3 Jitter
  5. 4 Pace
  6. 5 Share
  7. 6 Save
  8. 7 Pay

Take a step back. You cannot go faster than the limit. Ten per second is ten per second, and 200 companies take 20 seconds no matter how clever the retries are. The only thing you choose is how much of the budget you waste on requests that were never going to get through.

So instead of discovering the limit by bumping into it, you pace yourself. You know the limit from the documentation, and you send exactly that.

0102030limit: 10 per second0s10s20s30s40s50s60s
200 OK429 Too Many Requests
200 / 200
companies enriched
20 s
until the last one is done
0
calls rejected with a 429
Ten per second, never more. Zero 429s, and the fastest finish of all the strategies.

20 seconds, zero rejected calls. Twice as fast as step 3, and the API does not even notice you. That's what being a good citizen looks like: you use 100% of the budget you are given, and nothing more.

import Bottleneck from 'bottleneck'

// From the provider's documentation: 10 requests per second
const companyApi = new Bottleneck({
reservoir: 10,
reservoirRefreshAmount: 10,
reservoirRefreshInterval: 1_000,
maxConcurrent: 5,
})

for (const company of companies) {
companyApi.schedule(() => enrich(company)) // enrich() still uses callWithBackoff()
}

Bottleneck is one library among others, and a token bucket fits in 30 lines if you prefer to write your own. What matters is the behavior:

  • The rate is a setting, written next to the integration, with a link to the documentation it comes from.
  • Backoff stays, as a safety net. The limit can change, or someone else can use the key (more on that in a minute). 429s should become rare, not impossible.
  • Quota headers are an early warning. If RateLimit says 2 requests left and a reset in 40 seconds, slow down now, before the 429.
  • The 429 rate is a metric. With backoff in place, an occasional 429 breaks nothing. But it is a warning: everything that draws on this budget adds up to more than you are allowed. In a well-behaved integration the rate stays low. When it rises, something changed, and it deserves an alert someone can act on.

Step 5: you are not alone on the budget​

  1. 0 Unaware
  2. 1 Retry
  3. 2 Back off
  4. 3 Jitter
  5. 4 Pace
  6. 5 Share
  7. 6 Save
  8. 7 Pay

Your paced script now runs in production. A few minutes later, support reports that some customers get an error when they type their company number in the sign-up form.

The sign-up form uses the same API. With the same key. And so does a sync job that refreshes active accounts every ten minutes. Each consumer was perfectly paced to the full limit, as if it were alone. Together they ask for twice the budget, and the token bucket does not know that a customer is waiting: it rejects whoever arrives when the bucket is empty.

A rate limit never applies to a script. It applies to whatever the provider counts, and that depends on the API:

  • per API key, like in our example
  • per account, across every key your company owns
  • per user, when you call the API on behalf of a user with their own token
  • per IP address, often for unauthenticated calls
  • and often a mix: a limit per user, inside a bigger limit per account

Before sharing a budget, find out from the documentation what it is attached to. That tells you who you share it with, sometimes in places you did not expect: another team's integration with the same provider, or your staging environment.

03006009001200key limit: 600 per minute0 min5 min10 min15 min20 min25 min30 min
Customer-facing appSync job (every 10 min)Backfill scriptRejected (429)
30%
of customer requests rejected
12,867
backfill calls served in 30 min
Every consumer paces itself to the full limit, as if it were alone. Customers lose almost a third of their requests.

The fix is to treat the limit as one budget, shared by every consumer that counts against it:

  • One limiter per budget. An in-memory limiter is per process. If the backfill runs on 4 workers, each pacing to 10 per second, you send 40. The limiter has to live somewhere every process sees, usually Redis.
  • Priorities. A customer waiting in front of a screen comes first. Background work takes what is left.
  • Headroom. Keep 10 to 20% of the budget free. Customer traffic has peaks, and you don't want to discover them through an error page.
  • Separate budgets where you can. Production and staging should never share one, and a one-off script should never eat the budget of real customers. Many providers let you create several keys, but check that each key really gets its own limit: when the limit is per account, a new key is just a new name for the same budget.
03006009001200key limit: 600 per minute10% headroom kept free0 min5 min10 min15 min20 min25 min30 min
Customer-facing appSync job (every 10 min)Backfill scriptRejected (429)
0%
of customer requests rejected
8,747
backfill calls served in 30 min
One shared limiter. Customers first, the sync job next, the backfill takes what is left. Nobody gets a 429.

The backfill gets fewer calls, and takes longer. That's the point: it had no deadline, and the customers did. In code, it can be as simple as this (a sketch, any Redis-backed token bucket will do):

// One budget, stored in Redis, shared by every process that uses this key
const companyApi = sharedRateLimiter({key: 'company-api', perMinute: 540}) // 600 minus 10% headroom

// A customer is waiting: take a token as soon as there is one
await companyApi.take({priority: 'interactive'})

// Background work: only take a token while at least 30% of the budget is left
await companyApi.take({priority: 'background', keepFree: 0.3})

Step 6: spend the budget on useful calls​

  1. 0 Unaware
  2. 1 Retry
  3. 2 Back off
  4. 3 Jitter
  5. 4 Pace
  6. 5 Share
  7. 6 Save
  8. 7 Pay

Once all the traffic goes through one limiter, you can finally see where the budget goes. And the answer is often embarrassing. In our example, the sync job asks about the same 260 accounts every ten minutes: "did anything change?". Almost every answer is "no". A company changes its address or headcount maybe once a year.

That's polling: asking the same question again and again, in case the answer changed. It is the easiest thing to build and the most expensive thing to run.

Pollingevery 10 minutes"changed!"call #313Webhookthey call you"changed!"call #10h12h24h36h48h60h72h
One company, checked every 10 minutes. Its address changes once, after two days. Every grey line is a call that learned nothing.

Many APIs offer webhooks: you give them a URL, and they call you when something changes. One call per change, at the right moment, and it doesn't touch your budget. Instead of asking a thousand times, you are told once.

Webhooks come with their own homework:

  • Verify the signature. Anyone on the internet can call your URL. The provider signs each call, check it.
  • Answer fast, work later. Store the event, reply 200, process it in the background. If you take 30 seconds to answer, the provider considers it failed and sends it again.
  • Expect duplicates and gaps. The same event can arrive twice, so handle it idempotently. And some events will never arrive (your server was down, their queue had a bad day), so keep a slow reconciliation as a safety net. For example, once a night, ask the "what changed since yesterday" endpoint if the API has one.

Webhooks are the biggest win, but not the only one. The same question applies to every call: does it need to happen?

  • Cache what changes rarely. A company's legal name does not need a fresh API call on every page view.
  • Batch. If the API has an endpoint that takes 100 ids at once, one call replaces 100.
  • Fetch only what you use. If you call three endpoints per company and only display data from one, two thirds of your budget buys nothing.

In our example, the sync job goes from about 37,000 calls a day to a few dozen, plus one nightly reconciliation. That's budget handed back to everyone else who shares it.

The shortcut: "can't we just raise the limit?"​

At almost every step of this journey, someone asks the same question. "We have all these problems because the limit is too low. Why not pay for a higher one?"

Look back at the steps, and the answer is clear. A higher limit would not have fixed any of them:

  • Step 0. Promise.all over 50,000 companies asks for 50,000 requests at once. Unbounded concurrency eats any finite limit, it just takes a bit longer.
  • Step 1. A retry storm scales with the limit. With 100 times the budget, it fails 100 times faster.
  • Step 5. A shared budget stays shared. Without one budget and priorities, background jobs still take everything they can, and customers still lose. The noisy neighbor just gets a bigger house.
  • Step 6. Polling grows with your data. Twice as many accounts, twice as many useless calls, and you would pay more every year for the privilege of asking the same question.
Today: 10 per second99.9% rejected in the first second
10× limit: 100 per second99.8% rejected in the first second
100× limit: 1,000 per second98% rejected in the first second
Promise.all over 50,000 companies asks for 50,000 requests at once. Green is what the limit lets through.

There is also the other side of the table. The limit protects the provider's infrastructure, and your plan is not the only constraint. "Please let us send more requests we don't need" is not a strong opening, and they may simply say no.

Raising the limit for a client that does not respect it is buying a bigger bucket for a leaky pipe.

Step 7: the limit is now the business bottleneck​

  1. 0 Unaware
  2. 1 Retry
  3. 2 Back off
  4. 3 Jitter
  5. 4 Pace
  6. 5 Share
  7. 6 Save
  8. 7 Pay

So when is it right to pay? When you have done everything above, and it is still too slow for the business.

Say a marketing campaign brings 40,000 new leads at 9:00. Each lead needs three calls before a salesperson can call them: the company, its activity, its officers. Sales wants to start calling at 11:00, while the leads are still warm. Everything is in place: one shared limiter, priorities, webhooks, headroom, and only a handful of 429s. After live traffic and headroom, the background lane gets about 4 requests per second.

40,000
leads at 9:00
120,000
calls to make
8 h 20
to get through them at 4 per second
2 h
that sales can wait
040k80k120k11:00, sales starts calling17:2010:589:0011:0013:0015:0017:00calls left
Background share today: 4 per secondNeeded: 17 per second
40,000 new leads at 9:00, three calls each. Perfectly paced, zero 429s, and still six hours too slow.

This is the trigger. Not a 429, not an incident: everybody is a good citizen, the budget is fully and usefully used, and a business process still moves too slowly because of the pacing. The limit is no longer protecting the provider from you. It has become the bottleneck of the business.

And now the conversation with the provider is easy, because you come with a number and the reasons behind it:

"On campaign mornings, we need 17 requests per second for background work, on top of 4 for live traffic and a margin. Can we have 25? Here is our traffic: we use 90% of our budget, and our 429 rate has stayed under 0.1% for three months."

That's a request a provider can say yes to. It is a very different request from "your API keeps failing, please raise the limit", and it often gets a better price too. And sometimes the answer is something better than a higher limit: a bulk export, or a batch endpoint you did not know about.

  1. 1Few 429s in normal operation
    You pace yourself under the limit. The rare 429 is retried with backoff and breaks nothing, but you treat it as a warning: your consumers add up to more than you are allowed.
  2. 2No wasted calls
    Webhooks instead of polling, caching, batch endpoints, and you only fetch what you use.
  3. 3One budget, shared by everyone it counts
    Every consumer, process and pod that counts against the same limit goes through the same limiter, with priorities.
  4. 4The math still misses a business deadline
    With all of the above, real work waits longer than the business can afford.
  5. All four? Now buy a higher limit, and ask for a number.

The cheat sheet​

The whole journey, one line per step. Keep it next to the next API key you are handed.

  1. Read the limits first: how much, per what window, and counted against what.
  2. A 429 means "not now". Never retry immediately.
  3. Back off exponentially, with a cap.
  4. Add jitter, obey Retry-After, retry only what can heal, stop after a few attempts, and make writes idempotent.
  5. Pace yourself under the limit. Treat the rare 429 as a warning that the budget is overdrawn.
  6. One budget, one limiter. Shared by every script, process and pod that counts against it, with priorities and headroom.
  7. Spend the budget on useful calls. Webhooks over polling, caching, batching.
  8. Pay for a higher limit when all of the above is done and a business process still waits. Then ask for a number.

The API told you from the very first response: please slow down. Everything in this post is a way of listening.