OurSharedCode › Prompt Engineering Studio › Guides
For about three months I put think step by step in everything. It is one line, it is free to type, and the received wisdom says it makes models smarter. Then I sat down with a rewriting task and compared fifty outputs with the line and fifty without, and the difference was: the answers got longer, the calls got slower, and the quality was the same.
Chain-of-thought is a real technique with real gains. It is also the most over-applied idea in prompting, mine included, and the reason is that it never looks like it is doing harm. A prompt with a reasoning step produces output that reads as more considered whether or not it is.
The pattern is narrower than "hard tasks". Reasoning steps help where an early mistake would otherwise be invisible and would poison everything after it. Concretely:
Extraction. Lookup. Translation. Reformatting. Summarising. Rewriting for tone. Simple classification with clear categories. In all of these the answer is not the product of a chain — there is no intermediate result to get wrong — and asking for steps mostly buys you a preamble to strip out.
Two costs that are easy to miss when the technique is free to type. You pay for the reasoning tokens on every single call, forever, in latency as much as money. And the reasoning arrives mixed in with the answer, so whatever parses your output now has prose in front of it — which is a parsing problem you created, and I have written about how those end.
If you want reasoning without the mess, put a wall between the two parts and throw the first one away:
Work through the problem inside <thinking> tags.
Then give your final answer inside <answer> tags.
Everything outside <answer> will be discarded, so put nothing
there that the user needs.Here is the part that gets skipped, and it changes how much you should trust the technique.
The visible reasoning is generated text. It is not a transcript of a computation that happened somewhere else, and it is not guaranteed to be the reason the model produced the answer it produced. Models can and do write plausible steps that lead to an answer they were going to give anyway — this has been studied under the heading of faithfulness, and the finding is not comfortable: a chain of reasoning can look sound while the answer was driven by something the chain never mentions.
So: use chain-of-thought to improve answers, which it does. Do not use it as an audit trail and treat a convincing explanation as evidence the answer is right. Those are different claims, and only the first one is well supported.
The same caution applies to self-verification. Asking a model to check its own output catches formatting slips, arithmetic, and instructions it plainly missed. It is much weaker on factual errors, because the check runs on the same knowledge that produced the mistake — the blind spot is in both passes. A second call with fresh framing, or a check against a source, catches things a self-review will not.
This section has a shelf life, so take it as of late 2026. Models that do extended reasoning natively already work through problems before answering. Bolting think step by step onto one of those is usually redundant, and prescribing a rigid procedure can make things worse — you are constraining a process that was doing better unconstrained.
What still helps with those models is telling them what to think about: which constraints are hard, which trade-off you care about, what a wrong answer would cost. That is not the same as scripting the steps, and it is the part worth keeping.
The practical consequence is that a prompt tuned on last year's model may be carrying a reasoning instruction that is now dead weight. Worth checking when you switch models, along with everything else in the prompt.
Do not reason about whether reasoning helps. Measure it. Take twenty real inputs — twenty, not three — and run them with and without the line. Compare on the thing you actually care about, not on which output reads better, because the reasoning version will always read better.
If it wins, keep it and pay for it happily. If it draws, delete it and take the speed. That is a twenty-minute experiment which most prompts never get, and it is why so many production prompts are carrying a step that was cargo-culted in 2023 and never questioned since.
I keep the Think Step-by-Step block in the studio for the tasks that earn it, and I leave it out more often than I put it in.