How many tokens is 1,000 words?
For ordinary English prose, about 1,330 tokens. The usual rule of thumb is 0.75 words per token, or equivalently four characters per token.
That figure is correct and routinely misapplied. It describes English prose specifically, and most of what people actually send to a language model is not English prose. The same 1,000 words of source code runs closer to 2,600 tokens. As pretty-printed JSON, over 3,000. In a non-Latin script, 3,000 and up. The table above converts across all of them at once so the spread is visible rather than assumed away.
Why one number cannot cover it
A tokenizer is a learned vocabulary, not a formula. Byte-pair encoding starts from individual bytes and repeatedly merges the most frequent adjacent pair until it has a fixed-size vocabulary — around 200,000 entries for the current OpenAI encoding. Sequences that appeared often in the training corpus earn a single token. Everything else is rebuilt from fragments.
Since those corpora are dominated by English prose, English prose is what the vocabulary is efficient at. Common words are one token each. A UUID is a dozen. Deep indentation is one token per level per line. Non-Latin scripts frequently fall back to two or three tokens per character. None of that is a defect; it is simply what optimising for one distribution does to everything outside it.
Working backwards from a token budget
The conversion runs both ways, which is the more useful direction in practice. Given a 200,000 token context window, you can hold roughly 150,000 words of English prose — a full-length novel. The same window holds perhaps 75,000 words of source code, or 60,000 words of formatted JSON.
This is worth doing before you design around a window rather than after. Teams regularly plan a feature on the prose figure, build it against a corpus of structured data, and discover mid-implementation that the real capacity is half what they assumed. Confirm the actual fit in the context window calculator.
When to stop estimating
An estimate is fine for a sanity check, a rough capacity plan, or deciding whether an idea is in the right order of magnitude. It is not fine for:
- Sizing a prompt against a context window with little headroom.
- Committing to a monthly budget or a contract.
- Comparing two models where the difference is under about twenty percent.
- Anything involving code, JSON, or a language you did not measure.
In all of those, measure the real text. The token counter runs the actual encoder in your browser and takes about ten seconds, and it is exact for OpenAI models rather than approximately right. Precision is available; there is rarely a good reason to decline it.
Where these ratios come from
Each row is measured with o200k_base over representative samples of that content type, not copied from a blog post. They are averages, and the variance within a category is real — a densely commented codebase behaves more like prose, and a minified bundle behaves worse than the code row suggests. Treat them as a starting point, and see the methodology page for how everything on this site is derived.