The system prompt: the instructions you don't see
Before your first message even reaches the model, the app has already sent something: a block of instructions called the system prompt. It sets the model's persona, tone, rules and tools. It's why the same model can be a polite support agent in one app and a terse coding assistant in another.
What it looks like
Chat APIs take a list of messages, each with a role. The system prompt is the message with role system, placed first:
[
{"role": "system", "content": "You are a helpful assistant. Answer briefly."},
{"role": "user", "content": "What is the capital of Indonesia?"},
{"role": "assistant", "content": "Jakarta."},
{"role": "user", "content": "And its population?"}
]
It is sent with every request, because the model has no memory between messages (see the context window).
It's still just text
The model doesn't have a separate input for instructions. A chat template flattens the message list into one stream of tokens, with special markers showing where each role starts. For a Qwen model it looks roughly like this:
<|im_start|>system
You are a helpful assistant. Answer briefly.<|im_end|>
<|im_start|>user
What is the capital of Indonesia?<|im_end|>
<|im_start|>assistant
The system prompt has power only because models are trained to give text in that position more weight than user messages. It's a strong habit, not a hard rule.
What a system prompt is good for
- Persona and tone: "You are a friendly tutor for beginners."
- Format: "Answer in at most three sentences." "Reply in JSON."
- Rules and scope: "Only answer questions about cooking."
- Background facts: today's date, the user's name, product details.
- Tool instructions: the list of available tools and the exact format for calling them. In the playground, these are appended to your system prompt automatically.
What it can't do
- It isn't secret. Users can often get a model to repeat its instructions ("prompt leaking"). Don't put passwords or anything private in it.
- It isn't a security boundary. A cleverly written user message, or text inside a web page or tool result, can override it. This is called prompt injection. Real safety checks belong in code.
- Small models follow it loosely. A 1B-parameter model may drop a persona after a few messages or ignore a format rule. Larger models are far more consistent.
- It costs tokens every turn. A 2,000-token system prompt is paid for, and waited for, on every single message.
Writing a good system prompt
- Be specific. "Answer in 2–3 sentences for a 12-year-old" beats "be simple".
- Say what to do, not only what not to do.
- Show an example of the output you want. Models copy examples very well.
- Keep it short, especially for small models with small context windows.
- Test it. Ask the same question with different system prompts and compare.