TokenPad

Models

Instruction tuning

Instruction tuning is additional training that teaches a base model to follow instructions rather than merely continue text.

Also written: instruct model

A base model given "Write a poem about rain" may continue with more instructions, because that is what the training data looked like. An instruction-tuned model writes the poem.

Every model you reach through a chat API is instruction-tuned. It is also why boilerplate like "you are a helpful assistant" changes nothing — that behaviour is already in the weights.

In practice

It is why a modern model answers a question instead of continuing it as text. The practical consequence for prompting: instruction-tuned models respond to being told what to do, which means "answer in at most three sentences" works where "be brief" is left to interpretation. Write instructions as commands with checkable conditions.

Common questions

Why do models follow instructions at all?

Because they were trained to. A base model continues text; instruction tuning is the stage that turns "What is the capital of France?" from a sentence to extend into a question to answer.

How does this change how I write prompts?

Write commands with checkable conditions rather than descriptions. "Answer in at most three sentences" works where "be brief" is left to interpretation, and interpretation varies between runs.

Measure it

More in Models