TokenPad

Recipe · Reliability

Get reliable JSON out of a model

Constrain generation to a schema instead of asking politely, with a fallback extractor for providers that cannot.

The problem

You asked for JSON and got "Here is the result:" followed by a code fence followed by an offer of further help. Your parser fails on the first character.

The JSON inside is usually correct — this is a presentation problem, and it has a real fix rather than a repair function.

Constrain the output with a schematypescript
import OpenAI from 'openai';
import { z } from 'zod';
import { zodResponseFormat } from 'openai/helpers/zod';

const Triage = z.object({
  category: z.enum(['billing', 'technical', 'account']),
  confidence: z.enum(['high', 'medium', 'low']),
  reason: z.string(),
});

const client = new OpenAI();

// The provider enforces the shape during generation. This is enforcement
// rather than a request, and it removes the whole class of failure.
const completion = await client.chat.completions.parse({
  model: 'gpt-5.6-terra',
  messages: [{ role: 'user', content: ticket }],
  response_format: zodResponseFormat(Triage, 'triage'),
});

const result = completion.choices[0].message.parsed; // typed, validated
Fallback: extract JSON from a chatty responsetypescript
/** Last resort for providers or models without a structured mode. */
export function extractJson(raw: string): unknown {
  const trimmed = raw.trim();

  // A fenced block is the most common wrapper.
  const fenced = /```(?:json)?\s*([\s\S]*?)```/i.exec(trimmed);
  if (fenced) return JSON.parse(fenced[1].trim());

  // Otherwise take everything between the first brace and the last.
  const start = trimmed.search(/[{[]/);
  const end = Math.max(trimmed.lastIndexOf('}'), trimmed.lastIndexOf(']'));
  if (start !== -1 && end > start) return JSON.parse(trimmed.slice(start, end + 1));

  throw new SyntaxError('No JSON found in the response.');
}
The prompt section that works when a schema is unavailabletext
<output_format>
Return JSON only. The response must contain nothing before or after the object.

{"category": "billing" | "technical" | "account",
 "confidence": "high" | "medium" | "low",
 "reason": string}

Do not wrap the JSON in a code fence. Do not add commentary.
</output_format>

Why it is written this way

Enforcement beats instruction

Where a provider offers a structured output mode, generation is constrained to the schema and cannot produce anything else. That is categorically different from asking for a format and hoping, and it is usually a one-line change.

Why models add prose

Conversational helpfulness is trained in through preference optimisation. Counteracting it takes an explicit instruction that the response must contain nothing but the object — you are working against a learned tendency, not stating the obvious.

Validate keys, not just syntax

Valid JSON missing a required field breaks your application further downstream, where the cause is much harder to trace. Parse into a schema rather than into an untyped object.

What breaks the naive version

  • Growing the repair function indefinitely. The shapes a model invents when unconstrained are open-ended and your parser will keep chasing them.
  • Confusing truncation with malformation. "Unexpected end of JSON input" is a max_tokens problem, not a formatting one.
  • Using a very low temperature and assuming that guarantees format compliance. It reduces variation; it does not constrain structure.

Check your numbers