TokenPad

Parameters

Logprobs

Logprobs are the model’s reported confidence for each token it chose, and for the alternatives it considered.

Also written: log probabilities

They give you a usable confidence signal, which is otherwise absent — models express certainty in words regardless of how uncertain they are.

The main practical use is routing: low confidence on a cheap model escalates to an expensive one, which is what makes two-tier routing work without a separate classifier.

In practice

The useful application is a confidence signal you can route on: a classification returned with a low top-token probability is a candidate for human review or for a second pass on a stronger model. The caution is that these are the model's confidence in its own next token, not a calibrated probability that the answer is correct.

Common questions

What can I use logprobs for?

A routing signal. A classification returned with a low top-token probability is a candidate for human review or a second pass on a stronger model, which concentrates expensive attention where it is needed.

Are logprobs a measure of correctness?

No. They are the model's confidence in its own next token, not a calibrated probability that the answer is right. Confidently wrong answers have high logprobs.

Measure it

  • TemperatureTemperature scales the model’s output probabilities before sampling: below one sharpens the distribution towards the likeliest token, above one flattens it.
  • Top-pTop-p keeps only the most probable tokens that together make up a given share of the probability mass, and samples from those.
  • HallucinationA hallucination is model output that is fluent, confident and not supported by the source material or by fact.

More in Parameters