Model returned invalid JSON

What it means

Three different problems produce this symptom, and the fix differs for each. The model wrapped valid JSON in conversational text; the model produced malformed JSON; or the response was truncated mid-object.

The position in the error distinguishes them. Position 0 means a prose wrapper. "Unexpected end of input" means truncation. Anything in between is usually a genuine formatting mistake.

Why it happens

Most likely cause first.

1. 1

   Conversational wrapper

   "Here is the result you asked for:" followed by a fenced code block followed by "Let me know if you need anything else!". The JSON inside is usually perfect. Helpfulness is trained in deeply and an instruction has to counteract it.

2. 2

   Truncation

   Generation hit max_tokens mid-object. The output is not malformed so much as incomplete, and no amount of parser repair recovers the missing half.

3. 3

   Trailing commas or single quotes

   Both are legal in JavaScript object literals and illegal in JSON. Models produce them because their training data is full of both.

4. 4

   The schema was described vaguely

   If the prompt says "return JSON with the result", the model invents a shape — and a different one on the next call.

How to fix it

1. 1

   Extract and inspect what actually came back

   Strip the prose and the code fence, then check whether the JSON underneath is valid and has the keys you need. Usually it does, which tells you this is a presentation problem.

   LLM JSON Output ValidatorPull JSON out of a chatty answer and check the contract holds.
2. 2

   Rule out truncation

   Check the stop reason. If generation hit the ceiling, this is a max_tokens problem wearing a formatting problem’s clothes.

   max_tokens PlannerA ceiling derived from your p95, checked against window and quota.
3. 3

   Use structured output if your provider offers it

   It constrains generation to a schema rather than requesting a format. This removes the entire class of failure and is usually a one-line change.

4. 4

   Otherwise, tighten the output-format section

   Name the exact keys, show the shape, and state that the response must contain nothing but the object. Vague format instructions are the most common cause of a parser failing in production.

   Structured Prompt BuilderDeterministic, not AI-generated. Six sections, live token cost, copy as plain text or XML tags.

Stopping it happening again

• Validate against your required keys, not just that the JSON parses. Valid JSON missing a field breaks your application further downstream where it is harder to diagnose.
• Prefer structured output modes over instructions wherever available — enforcement beats request.
• Do not grow repair logic indefinitely. The shapes a model invents when unconstrained are open-ended, and your parser will keep chasing them.

Frequently asked questions

Why does the model add text when I asked for JSON only?

Conversational helpfulness is trained in through preference optimisation. Counteracting it takes an explicit instruction that the response must contain nothing but the object — or a structured output mode, which enforces it.

Is it safe to just strip everything outside the braces?

As a safety net, yes. As the primary approach, no — it will silently do the wrong thing when the response contains braces in prose, and it hides a prompt problem you could fix once.

Related errors

• Response cut off mid-sentencellm response cut off incomplete max_tokens truncated
• Invalid request (400)invalid_request_error 400 bad request openai anthropic