Asking a Weaker Question First to Get a Better Answer
The technique is called step-back prompting, and its whole shape is one sentence: two calls, where the first answer becomes the second call's context.
Style: notebook-3d • hero generated with gpt-image-2
Core Concept
You have a hard, detail-heavy question. The instinct is to ask it directly. Step-back prompting says: don't. First ask a deliberately weaker, more general question — what principle governs this kind of problem? — get that answer, then ask your real question with that principle sitting in the context window.
It works for a mechanical reason, not a mystical one. Retrieval and attention are both driven by the tokens in front of them. The abstract question isolates the fundamental concept from the noisy details and pulls the governing rule into context; the concrete question then reasons from a stated rule instead of from a half-remembered instance.
Source: arXiv 2310.06117v2 — Take a Step Back: Evoking Reasoning via
Abstraction in Large Language Models, Zheng et al., Google DeepMind.
Key Components
The step-back question
A generalisation of the original prompt. Not a rephrasing and not a hint — it asks for the class of principle the question belongs to.
The grounding answer
The model's answer to that weaker question. This is the artifact you actually wanted: a principle stated in tokens rather than recalled implicitly.
The second call
The original prompt again, now with the principle prepended as context. Two calls, not one — that extra round trip is the cost.
The composition point
What genuinely composes with step-back is retrieval — Step-Back + RAG
is its own arm in the paper. Chain-of-thought is a competing baseline, not a partner.
How It Works
- Complex prompt. The real question, full of specific detail — dates, entities, quantities, edge conditions. generalize →
- Step-back question. "What general principle or fact does this question depend on?" Deliberately weaker and more abstract. answer →
- Step-back answer. The stated principle. Cheap to verify, and you can read it — which is itself a debugging win. use as grounding context →
- Final answer. The original prompt re-asked on top of that grounding. The reasoning now starts from a rule instead of from the question's surface.
Versus chain-of-thought: CoT reasons forward step by step from
the question's own surface details, so a wrong early step poisons the whole chain. Step-back
retrieves the governing principle first and reasons down from it. The paper reports gains
of up to 36% over CoT and lists PaLM-2L + CoT and
PaLM-2L + Step-Back as separate arms.
What It Actually Bought (PaLM-2L)
| Task | Baseline → Step-back | Gain |
|---|---|---|
| MMLU Physics | 66.4 → 73.2 | +6.8 pts |
| MMLU Chemistry | 70.9 → 81.8 | +10.9 pts |
| TimeQA | 41.5 → 66.0 | +24.5 pts |
| TimeQA + RAG | 41.5 → 68.7 | +27.2 pts |
| MuSiQue | 35.5 → 42.6 | +7.1 pts |
These are percentage POINTS, not percent. Say "+24.5 points" or "four in ten to two in three on TimeQA". Saying "+27%" invites the relative reading of +65% — about 2.4× the real effect. All three of PaLM-2L, GPT-4 and Llama2-70B were tested; these five rows are PaLM-2L's.
TimeQA is the striking row and it fits the mechanism: multi-hop, time-scoped questions are exactly where surface details mislead and a retrieved principle helps most.
Where it loses — from the paper's own tables
- Llama2-70B MMLU Chemistry: CoT 1-shot 68.1 vs step-back 66.7.
- PaLM-2L StrategyQA: CoT 83.6 vs step-back 82.7.
- PaLM-2L GSM8K: CoT 84.4 vs step-back 84.3 — the authors concede the principles here are so simple that retrieving them first is "not absolutely necessary".
- It costs a whole extra round trip. On simple lookups a vague principle can displace a detail that was already retrievable.
- It cannot rescue a question whose facts were never in the corpus.
The test: run the same evaluation set both ways and report where it LOST, not only where it won.
Real-World Applications
- Multi-hop / time-scoped QA. The TimeQA and MuSiQue shape — "which role did X hold during Y?" Ask for the timeline principle first.
- RAG pipelines. Use the step-back answer as the retrieval query. This is the arm that actually composes, and it was the best TimeQA number in the paper.
- Physics and chemistry word problems. "Which law applies?" before "compute the value" — the two biggest MMLU gains.
- Code and config debugging. "What invariant is this system supposed to maintain?" before "why is this request failing?"
- Where to NOT reach for it: single-fact lookups, arithmetic like GSM8K, and any latency-sensitive path where a second round trip is the budget.
Checkpoint — answer before you move on
- Name the four steps of the cascade and the verb on each edge. Why must the step-back answer arrive before the second call rather than after it? Hint: what the tokens in the context window are doing for attention and retrieval.
- TimeQA went 41.5 → 66.0. State that gain in a way that cannot be misread, and explain why "+27%" would overstate it — and what the 27 figure actually requires. Hint: points vs relative percent; and one arm in that table isn't step-back alone.
- Give one benchmark where chain-of-thought beat step-back, and say what property of that benchmark explains the loss. Hint: three rows qualify. The authors explain one of them in a single clause.