OurSharedCode › Prompt Engineering Studio › Guides
I once deleted about two hundred words of instructions from a prompt and replaced them with two examples. The output got better immediately, which was satisfying, and it stayed subtly wrong for a fortnight before I worked out why: both my examples happened to be short. Every answer I got back was short, including the ones that needed not to be. I had not written that rule anywhere. I had demonstrated it twice, which turned out to be stronger.
That is the thing to hold on to about few-shot prompting. Examples do not illustrate your instructions — they compete with them, and they usually win. Everything below follows from that.
For teaching a format, two is usually enough and three is comfortable. Format is a shallow pattern and models pick it up almost immediately; the fourth example of the same JSON shape teaches nothing the first three did not.
For teaching a judgement — which of eight categories this belongs in, where your line is between "urgent" and "normal" — more helps, because you are not showing a shape, you are showing a boundary. That needs examples on both sides of it.
Every example is in every call, so a long example set is a permanent tax on latency and cost. That is the real ceiling, and it is why I stop adding examples the moment a new one stops changing the output. Add one, run your test set, keep it only if it earned its place.
The instinct is to pick clean, typical, representative cases. It is the wrong instinct. Typical cases are the ones the model already handles; you are spending your budget on the part of the task that was never in doubt.
Pick the awkward ones. The ticket that mentions two problems and only one of them matters. The review that is sarcastic. The invoice with the date in the wrong format. The case that made you stop and think about which answer was right — if you had to think, so will the model, and that is exactly where a demonstration pays.
One caveat, learned the hard way: if every example is a hard edge case, the model concludes the task is generally hairy and starts finding complications in simple inputs. A mix, weighted towards the awkward, beats either extreme.
This is the part that actually bites, and my short-answers fortnight was only the most obvious version. A model has no way to know which properties of your examples were the point and which were incidental. Every property they share is a candidate rule.
approved, expect
approved to come back more than it should. Balance the
labels even when your real data is unbalanced.
None of these are model defects. They are the mechanism working exactly as advertised on input that had more patterns in it than you intended.
If the live input arrives wrapped in <ticket> tags, wrap
the examples in <ticket> tags. If the real thing has a
subject line, give the examples subject lines. Any difference between the
shape of your examples and the shape of the real input is a difference the
model has to bridge, and it will bridge it in whichever direction it
happens to guess.
<ticket>
Subject: card declined twice
I tried paying this morning and it bounced, then again at lunch.
The bank says there is nothing wrong at their end.
</ticket>
<output>
{"category": "billing", "urgency": "high", "needs_human": true}
</output>
<ticket>
Subject: how do I change my avatar
Hi — can't find where to upload a new profile picture. No rush.
</ticket>
<output>
{"category": "account", "urgency": "low", "needs_human": false}
</output>
Note that those two disagree on every field. That is not decoration — it is
the point. Two examples that both come back
billing / high / true teach the shape of the JSON and nothing
about how to decide.
When a task genuinely requires reasoning, showing only inputs and finished answers teaches the model to jump. It has seen two problems solved in one move, so it makes one move, and on the third problem the move is wrong.
Either include the reasoning inside the example — show the working, then the answer, in the shape you want it — or leave examples out of that part of the prompt entirely and let the model reason. Which of those is right depends on whether the task really needs steps, and I have written about that in chain-of-thought is oversold.
Examples also go stale in a way prose does not. An example containing a product name, a price, a status code or a category you later renamed will keep teaching the old world long after you updated the documentation. Examples are code. They belong under review and they need an owner.
Swap the order and rerun. If reordering your examples moves the output much, the model is leaning on their arrangement rather than their content, and you are over-fitted to an accident. Rebalance them.
Feed an example back as live input. Take example two, remove it from the prompt, and send it in as the real thing. You should get back approximately what you wrote as its output. If you do not, your examples are not internally consistent with each other — one of them is teaching a rule the other one breaks, and that is worth finding before it shows up as flakiness in production.