What is a RAG feedback loop?

User signals sharpen the ranker — if read honestly.

The loop, three moves

• Log: which passages were retrieved, cited, clicked, how long dwell, copied, rejected, thumbs, corrections.
• Grade: turn those events into relevance judgments per (query, passage) pair.
• Update the ranker: a learned re-ranker from click comparisons, a contrastive fine-tune of the embeddings, or cheaper — promote/demote document weights, curate the corpus, rewrite the chunking.
• Generation-side corrections (“no, it is X”) feed a different consumer: fine-tuning data or a verified-answers cache — not the retriever.
• Without the loop, retrieval quality is frozen at whatever the offline benchmark said on day one.

Signals, read honestly

• Click means examined, not agreed: position bias, presentation bias, curiosity bias all ride in the click.
• Dwell time is two-valued — long dwell means engaged or confused; pair it with a companion signal (returned? copied?) to disambiguate.
• Copy/adoption is the strongest satisfaction proxy: the answer went into the user’s own work.
• Thumbs are sparse and self-selecting, skewed to the angry: high-quality, low-coverage — label a golden set, don’t train directly on them.

The bias trap

• Rank-1 gets clicked whether or not it is best — the classic examination–position effect.
• Naive click-training teaches the ranker to be clickbait: it learns “what gets examined,” not “what answers.”
• Fix: explicit debiasing — inverse-propensity weighting from an examination model, or within-session comparisons where position cancels.
• A loop trained on its own clicks drifts rich-get-richer: add exploration (rotate a candidate in) or periodic re-anchoring to human-labeled sets, or the system self-confirms its ranking.

The honest test

• Run the loop on synthetic traffic where true relevance is known, but inject position bias: rank-1 gets +30% click probability regardless of quality.
• A naive click-trained ranker must converge toward the biased ranking — accuracy drops versus the oracle.
• The propensity-debiased variant should recover the oracle ranking.
• If both variants match, the bias story is decorative — that gap is the lesson.