Parameters
Truncation
Truncation is generation stopping because a limit was reached rather than because the model finished.
It happens when max_tokens is hit or when input plus requested output exceeds the context window. On newer models the request succeeds and simply stops, which is far harder to notice than an error.
The response carries a stop reason distinguishing "finished" from "hit the limit". Checking it is a few lines and turns a silent product defect into a handled case.
In practice
Truncation arrives as success. HTTP 200, a well-formed response, an answer that stops mid-sentence — and nothing in the transport says anything is wrong. Checking the stop reason on every response is the single highest-value line of error handling in an LLM application, and it is the one most often absent.
Common questions
How do I detect a truncated response?
Check the stop reason on every response. A truncated answer returns HTTP 200 with well-formed content that stops mid-sentence, and nothing else in the response indicates a problem.
What should I do when a response is truncated?
Treat it as a failure, not an answer. Either raise max_tokens, ask for a shorter answer, or continue the generation deliberately — but never render it to a user as if it were complete.