LLM & Inference • Sensei Learning System

The Difference Between Telling And Showing A Model What You Want

Zero-shot vs. few-shot: steering a model at prompt time — no weights change, ever.

Notebook infographic: telling a model (zero-shot) vs showing a model (few-shot), with the GPT-3 SuperGLUE accuracy curve and a three-step workflow

Core Concept

Every prompt is an instruction, but a model's interpretation of that instruction can drift in ways the words alone can't control. Telling (zero-shot) hands it only the instruction — the model guesses the answer's shape from its pretrained distribution. Showing (few-shot) adds a few input→output examples inside the prompt. The weights never move; the examples act as in-context demonstrations that pull the output toward the pattern the examples share. That is in-context learning — inferring a task spec from samples, not from a description.

Key Components

Zero-shot (telling)Instruction only, no examples. Works when the task is conventional (translate, summarise, classic categories) — and fails quietly when the expected format is non-obvious: a plausible answer in the wrong shape.
Few-shot (showing)2–5 concrete input→output pairs inside the prompt. Not training data — no gradient, no weight update. They constrain the output distribution toward a shared pattern.
The price tagExamples cost only input tokens — not fine-tuning compute. So the trade is prompt length (latency, cost) against output quality, with a hard context-window ceiling.
The two trapsDiminishing returns: past a task-dependent threshold, more examples plateau or degrade (they crowd the query out of attention). And ordering matters — a different sequence of the same examples can move accuracy 10+ points (Lu et al. 2022).
Brown et al. (GPT-3, 2020), SuperGLUE: 52.0 at 0-shot → 55.4 at 1-shot (+3.4) → 71.8 at 32-shot (+19.8). Most of the gain arrives with the first handful of examples — and it's paid in tokens, not training.

How It Works — the practice loop

  1. Start zero-shot on any new task. It's the cheapest, fastest baseline and it tells you whether the model's default guess matches the shape you need.
  2. If the shape is wrong, show, don't re-tell. Add 2–5 examples that specifically demonstrate the failure mode — the label set, the JSON fields, the tone — rather than writing a longer, more careful instruction.
  3. Stop at the plateau. Watch quality as you add examples. Past the threshold it stops improving and can worsen; re-order or trim instead of stacking more.
Adjacent but distinct — don't conflate: chain-of-thought adds reasoning steps, not input-output pairs. Fine-tuning (LoRA, full) changes the weights — a different cost/capability curve. RAG puts documents in context, but they are evidence, not task demonstrations.

Where You'll Meet It

Custom label sets that aren't in the model's vocabulary Structured extraction — a strict JSON shape Consistent house tone across support replies Domain tasks where the default format is off

Checkpoint — answer before you move on

  1. A model asked to sort tickets into your 3 custom labels keeps inventing new labels and mixing in JSON. Zero-shot is failing. What's the cheapest fix, and exactly what do you add? Stop rewriting the instruction. Show 2–5 input→output pairs using your exact label set.
  2. Few-shot examples cost zero in weight updates. So what are the examples actually doing to the model while it runs? In-context learning: the model infers a task specification from the pattern the examples share.
  3. You added your 12th example and accuracy went down from the 5th. What are the two mechanisms that can cause that? The plateau/crowding effect, and example ordering — sequence can shift accuracy 10+ points.