Topic 3.1
Prompts Are Specifications
In one line
A prompt is the requirements document for a very literal worker. A vague prompt gets a vague answer; a prompt that states the audience, length and format gets output your program can rely on.
Think of it like this
Asking a taxi driver to "take me somewhere nice" vs giving the address. The driver is skilled, but with no destination you get a random place and an argument. With an address, a time and "please avoid the highway", you arrive where you wanted. A model is the same: very capable, but it only knows what is written in the prompt.
Words you'll meet
New words in this topic, in plain English. Come back here whenever one feels fuzzy.
- Prompt
- The text you send to a model: your instructions, the data to work on, and any examples.
- Specification (spec)
- A written description of exactly what a piece of software must do. A prompt plays this role for a model.
- Fake model
- A small scripted function in these examples that returns replies depending on the prompt, so the mechanics run for free and give the same result every time.
- Checkable requirement
- A requirement a program can verify, like "at most 10 words". Opposite of a wish like "keep it short".
- Output
- The text the model sends back.
Step by step
01Step 1: start with the obvious prompt
"Summarise this email." It works in a demo, and the reply looks fine to a human. But we have a product need: the summary appears in a small box on the agent's screen and must be at most 10 words.
02Step 2: make the need checkable
Turn "short" into code: count the words of the reply and compare with 10. Now a prompt either passes or fails. The first example prints this verdict for three prompts.
def passes(reply):
return len(reply.split()) <= 10
print(passes("Late parcel; refund wanted."))03Step 3: add the missing facts, one at a time
The vague prompt fails (a long general overview). Adding the audience ("for a support agent") changes the content but not the length: still a fail. Adding "Reply in at most 10 words" finally passes. Each gap you close changes the output in a way you can measure.
04Step 4: the fake model teaches the lesson
The fake model in the example is only a few lines of "if the prompt contains X, reply Y". A real model is far more flexible, but the principle is the same: the reply is a function of the prompt. Change the words, change the result.
05Step 5: keep the spec, not the story
The final prompt is not clever. It is a plain list of requirements. Keep it in your code with a comment saying what each sentence is for, so the next person knows which sentences are load-bearing.
Try it yourself
- 1
Predict first
Before pressing Run, write down which of the three prompts you expect to pass the 10-word check. Then run and compare.
- 2
Add a new requirement
Change the check to "at most 6 words" and run. All three fail now, because the fake model has only one short reply. In real life you would now edit the prompt ("at most 6 words") and the model would adapt. The fake model only knows the 10-word rule, which is a reminder that a fake is a stand-in.
- 3
Compare languages
Run the example in all three languages. The prompts, replies and verdicts must match exactly. Find the line in each language that counts the words.
Code & diagrams
A fake model answers differently depending on what the prompt says. A one-line check counts the words.
Expected output
vague: 36 words, FAIL
This is an email from a customer. It mentions a parcel, a delay and some feelings. There is a lot that could be said, so here is a general overview of what the email is about.
with audience: 24 words, FAIL
The customer says the parcel is two weeks late and asks for a refund or a new one. They sound upset about the wait.
full spec: 8 words, pass
Late parcel; customer wants a refund or resend.Break it on purpose
Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.
Break #1
Change one word of the spec
In the "full spec" prompt, change "at most 10 words" to "at most ten words" (the fake model looks for the exact digits).
Myth vs fact
Myth
A smart model understands what I mean.
Fact
It only has your words. It guesses the rest from common patterns, so what you leave out is filled with the most average answer.
Myth
A longer prompt is always a better prompt.
Fact
Length is not the goal; completeness is. Add the missing requirements, remove sentences that do nothing, and measure. Every extra token also costs money on every call (Topic 3.5).
Myth
If the output looks right, the prompt is right.
Fact
One good output proves almost nothing. A prompt is right when it passes a check over many inputs (Topic 3.8).
Pro corner
Extra depth for experienced readers. New to this? Skip it for now and come back later.
- ▸
Put the requirement into code twice: once as the sentence in the prompt, once as a validator on the output (word count, allowed labels, JSON shape). The prompt asks, the validator verifies. Never rely on the first alone (Phase 4).
- ▸
Prompts are shared state. Several features reading the same prompt string means one edit changes all of them. Give each use its own named prompt with a version (Topic 3.8).
- ▸
Write down the reason for each sentence in a comment or the changelog. Months later, "why does it say never use bullet points?" has an answer instead of a fear of deleting it.
Remember this
- 1
Why this matters. A prompt is the text you send to the model. The model cannot ask you what you meant, cannot see your screen, and has never met your customers. Everything it knows about your task is in the prompt (plus what it learned in training). When the output is wrong, the first question is not "is the model bad?" but "did my prompt say what I wanted?".
- 2
A prompt is a specification. Software engineers know the rule: if the requirement is unclear, the code will be wrong however good the programmer is. A prompt is the requirement. A good one answers four questions: what is the task, who is it for, what shape must the answer have, and what must it not do.
- 3
Vague in, vague out. "Summarise this email." leaves everything open: how long, for whom, which details matter. The model fills the gaps with its most common guess, which is a long, polite, general overview. That is fine for a person reading it once. It is useless for a program that must show 10 words in a small box.
- 4
Make the requirement checkable. A spec you cannot check is a wish. "Short" is a wish; "at most 10 words" is a test you can run in one line of code. Write prompts so that some code can say pass or fail. This habit is what makes prompts testable later (Topic 3.8) and is the base of evaluations (Phase 9).
- 5
Fix one gap at a time. The example below starts with the vague prompt and fixes one gap per step: first the audience, then the length. Each time you change the prompt and measure the result, not just read it. Without measuring, you cannot tell a better prompt from a luckier run.
- 6
A prompt is not magic words. Online lists of "secret phrases" mostly copy habits that worked on an old model once. Clear, specific, complete instructions work on every model and age well. Chasing tricks is the part of prompting that rots; writing specifications is the part that lasts.
Explain it without notes
Why should you think of a prompt as a specification rather than a question?
What makes a requirement in a prompt "checkable", and why does it matter?
A model's summary is too long. List three things you try before blaming the model.
Practice
Write short_enough(reply, max_words) that returns True when the reply has at most max_words words. Test it with a 3-word and a 12-word reply and max_words=10.
Write spec_check(reply) that returns a list of failed requirements for a reply that must be at most 10 words and must not contain the word "overview". Return an empty list when everything passes.
Trade-offs
- ↔
A detailed spec gives predictable output but costs more input tokens on every call; trim sentences that do not change the output.
- ↔
Strict limits (word counts) make output easy to check but can cut useful detail; choose limits from the real product need.
- ↔
Tuning a prompt by hand is quick for one case but hides regressions; a small test set (Topic 3.8) costs setup time and saves it later.
Done when you can
Done when you can explain why a prompt is a specification.
Done when you can turn a wish ("short") into a checkable requirement ("at most 10 words").
Done when you can fix a vague prompt one gap at a time and measure each change.
Done when you can write a validator that tests a prompt's requirements on the output.