What this tool tells you
Rate limits are the constraint that breaks a launch, not the one that shows up on an invoice. Enter your peak traffic and your quota, and this shows which of the two limits binds first, how much headroom is left, and how many concurrent workers the throughput needs.
Two quotas, and they are not the same
Providers enforce at least two limits simultaneously, and you are bound by whichever you hit first:
- Tokens per minute (TPM). Usually the binding one for anything with substantial prompts. A retrieval-augmented endpoint sending 8,000 tokens per request exhausts a two-million TPM quota at 250 requests per minute, well before any request limit matters.
- Requests per minute (RPM). Binds for high-volume, small-prompt traffic — classification, routing, embedding lookups.
Neither has anything to do with the context window, which limits the size of one request rather than your aggregate throughput. If those terms blur together, read context window vs token limit vs max tokens.
The max_tokens trap
Several providers count your requested max_tokens against the quota rather than what the model actually generated. If you set it to the model maximum "to be safe", you may be reserving tens of thousands of tokens of quota on a request that returns two hundred.
The toggle above models this. If it applies to your provider, setting a realistic ceiling can multiply your effective throughput without changing a line of application logic — one of the rare changes that is pure upside.
Plan against the peak, not the average
The most common error in this arithmetic is dividing a monthly volume by the number of minutes in a month. Traffic is not uniform: a batch job at nine in the morning, a marketing send, or a single enterprise customer running an import will produce a minute that is twenty times the mean.
Take your peak minute from production metrics. If you do not have production yet, assume peak is five to ten times your average and design for that. The average never breaks anything; the peak breaks everything.
Concurrency
The workers figure is throughput times latency: to sustain 600 requests per minute at four seconds each you need forty in flight simultaneously. Two consequences follow.
First, slower models need more concurrency for the same throughput, which is a real infrastructure cost that never appears in a price comparison. Second, every one of those workers retries independently when a 429 arrives — which is why naive retry logic turns a brief overage into a sustained outage. Exponential backoff with jitter, always.
When you do not fit
- Reduce tokens per request. Directly raises your TPM ceiling. The prompt optimizer and data format optimizer both convert straight into throughput here, not just savings.
- Lower max_tokens if your provider counts the ceiling.
- Spread load across models. Quotas are typically per model, so routing simple work to a cheaper model frees quota on the expensive one — throughput and cost improving together. Shortlist candidates in the model finder.
- Queue what is not urgent. Anything not blocking a user can wait, and batch endpoints usually sit outside the interactive quota entirely — at roughly half the price.
- Raise the tier. Usually a function of spend and account age, so it is the slowest lever rather than the first one.
Once throughput fits, put the same traffic numbers into the cost calculator to see what sustaining it costs per month.