TokenPad

Methodology

Every number on this site is either measured or derived, and this page says which is which. If a figure here cannot be traced to a provider’s own documentation or to code you can inspect, it should not be on the site.

Token counts

Tokenization runs entirely in your browser using the gpt-tokenizer implementation of OpenAI’s byte-pair encodings. Two encodings are loaded on demand: o200k_base, used by GPT-4o and everything after it, and cl100k_base, used by GPT-4 and GPT-3.5. Neither is bundled into the initial page load, so a visitor who reads the guide without using the tool never downloads them.

For OpenAI models this produces an exact count: the same encoder the API uses, run over the same bytes. No approximation is involved and none is claimed.

Why other providers show estimates

Anthropic, Google and DeepSeek do not offer a tokenizer that can run in a browser. Anthropic exposes token counting only through a server-side API endpoint, which this site cannot call without becoming something other than a free static page. Google’s Gemini models use a SentencePiece vocabulary with no published browser build. DeepSeek ships a Hugging Face tokenizer that is not bundled here yet.

Faced with that, there are three options. Omit those models entirely, which makes the tool much less useful. Show a scaled tiktoken count and call it a token count, which is what most comparable sites do and which is quietly misleading. Or show the scaled count, label it an estimate everywhere it appears, and publish the scaling factor. This site takes the third option.

The factors, and where they come from

  • Anthropic does not publish a client-side tokenizer. Counted with o200k_base, then scaled: ~1.18x for the historical tiktoken/Claude gap, times ~1.30x for the newer tokenizer introduced with Claude 4.7.
  • Anthropic does not publish a client-side tokenizer. Counted with o200k_base and scaled ~1.18x, the commonly reported gap between tiktoken and Claude's pre-4.7 tokenizer.
  • Gemini uses a SentencePiece tokenizer that has no browser build. Counted with o200k_base at parity; no published conversion factor exists, so treat this as an order-of-magnitude figure.
  • DeepSeek ships a Hugging Face tokenizer that is not bundled here yet. Counted with o200k_base at parity pending a real browser build.
  • xAI does not publish a client-side tokenizer. Counted with o200k_base and scaled slightly; the direction of the error is known but not its size, so treat this as an order-of-magnitude figure rather than a count.
  • Mistral publishes its Tekken tokenizer only as a Python package, with no browser build. Counted with o200k_base and scaled; Tekken generally produces more tokens than o200k_base on the same English text.

These are approximations of a real quantity, not measurements of it. Treat them as accurate to within roughly ten to twenty percent, and never as the basis for a decision that turns on the last few percent — sizing a prompt right at a context window boundary, for instance. Where precision at that level matters, use the provider’s own token counting endpoint from your backend.

When a browser-capable tokenizer becomes available for any of these providers, it replaces the estimate and the badge changes. That is a deliberate design goal of the data model rather than an afterthought.

Prices

Every model entry stores the URL it was read from and the date it was read. Both are published in the model price table, and rows older than 45 days are flagged there automatically so staleness is visible rather than hidden. The current data set was last reviewed in full on August 5, 2026.

Prices are never copied from an aggregator, a blog post or from memory. They come from the provider’s own pricing documentation, and a price change and its source date are always updated together in the same edit. Prices quoted are standard on-demand rates in US dollars per million tokens; batch discounts, negotiated enterprise rates and regional premiums are out of scope and are stated as such on the tools that would otherwise imply them.

Context windows

A context window appears in the context window calculator only when the provider documents it plainly. Where a figure is ambiguous or undocumented, the model is left out of that tool rather than assigned a plausible number. An omission is honest; an invented figure in a tool people use to size production prompts is not.

Corrections

If a number here is wrong, it is a bug and it will be fixed. Follow the source link on the row in question, and if the provider disagrees with us, tell us — the provider is right.