Semi-Supervised Learning: When Unlabeled Data Helps, and When It Backfires

A few labels anchor the geometry. The unlabeled mass supplies the shape.

• Section 1 - The regime boundary:
• Supervised: only labels, nothing else
• Self-supervised: zero labels, invented task
• SSL: a few labels pin the class shape
• The mass supplies the geometry
• The boundary is the identity: some, not none

• Section 2 - The only-if: three assumptions:
• Locality: nearby points share labels
• Cluster: boundaries pass through sparse regions
• Manifold: features ride a low-dim shape
• None of the three is guaranteed
• Fail one, and the mass becomes poison

Section 3 - Five methods, one trade-off:

- Pseudo-labeling: label your own data - Consistency: two views, one prediction - Graph: labels spread over connected points - Teacher-student: EMA teacher, student learns - Entropy: sharpen the unlabeled predictions Simple methods compound their own errors Stable ones cost O(N^2) graphs or a 2x teacher

• Section 4 - Error amplification:
• Confident for the wrong reasons
• Wrong pseudo-labels lock in and reinforce
• Held-out test: below baseline, the loop harms
• Minority classes wash out - report per-class F1
• Domain shift: mass from a different process
• Shifted-domain test: must not beat labeled-only

Sticky note - the whole lesson:

Unlabeled data helps when structure matches the task.