TokenPad
RAG

Reranker Cost Estimator

Work out whether reranking pays for itself, before the accuracy argument.

Settings
Reranker CostExact
0Token change
Token change0no change
Input tokens0what you pasted
Output tokens0what you would send
Token cost of this result
Output tokens0
As input$0.00
× 100K requests$0.00

Everything on this page runs in your browser. Nothing you paste is transmitted, because there is no server here to transmit it to.

Result
 

The trade

Reranking adds a per-query fee and removes generation tokens, because you send four good chunks instead of twelve mediocre ones. Whether that nets out positive depends on your chunk size and your generation price.

At typical numbers it does, because generation input costs far more per token than reranking costs per query. The calculation is worth doing rather than assuming in either direction.

The stronger argument

Even where reranking loses on cost, it usually wins on quality. Irrelevant context measurably degrades answers, so cutting from twelve chunks to four often improves output at the same time as reducing input.

If the numbers here come out negative, justify the reranker on accuracy and treat the fee as a quality cost. Do not justify it on a saving that is not there.

Frequently asked questions

How many candidates should I rerank?
Wide enough that the right passage is in the set — commonly thirty to a hundred. Reranking is cheap per query and the whole point is to search broadly and then be selective.
Can I use a small LLM as a reranker?
You can, and it is usually more expensive per query than a dedicated reranking model while being slower. Compare against the rate here before choosing that path.

More rag tools