Recurrent Neural Network
Networks that remember — built for sequences
🧠 Core Concept
An RNN is a neural network with a loop. At each step it reads one element of a sequence and blends it with a hidden state — a running memory of everything seen so far. That memory is what lets it model order and context, so word 5 can depend on word 1.
🧩 Key Components
Hidden State
A vector carried forward each step — the network's short-term memory of the sequence so far.
Unrolling Through Time
The single looped cell is "unfolded" into a chain, one copy per time step, sharing the same weights.
BPTT
Backpropagation Through Time: gradients flow backward across every unrolled step to update weights.
Vanishing Gradients
Repeated multiplication shrinks (or blows up) gradients over long sequences, erasing long-range memory.
⚙️ How It Works
Start with an initial hidden state h₀ (usually zeros).
At each step t, combine the new input xₜ with the previous memory hₜ₋₁ and squash through an activation.
The result becomes the new hidden state hₜ — and can produce an output yₜ.
Repeat down the whole sequence, then train with BPTT to nudge the shared weights.