02 · Neural Networks & Model Architectures · Topic 04

What are RNNs and LSTMs?

RNNs and LSTMs are neural-network architectures designed to process sequences step by step.

What is it?

RNN means Recurrent Neural Network: it carries information from earlier sequence steps into later processing.

LSTM means Long Short-Term Memory. It is a type of recurrent network designed to retain useful information across longer sequences more effectively than a basic RNN.

Think of it like...

Reading a sentence one word at a time while carrying notes from earlier words.

Those notes give the next step some context, so the network can use what came before without rereading the whole sequence.

How it works

The network reads one item at a time and updates its hidden state. LSTMs add gating mechanisms that help decide what information to keep or forget.

1 · SequenceReceive ordered dataWords, time-series values, or sound frames arrive in order.

2 · Step 1Read the first itemThe network processes the current item.

3 · Hidden stateCarry contextInformation from earlier steps is stored in a hidden state.

4 · Step 2Update the stateThe next item is processed together with the carried context.

5 · OutputContinue or respondThe updated state supports a prediction or final output.

Real-world examples

Earlier language models

Input: Words in sequence

System: Carries context from earlier words

Output: A predicted next word

Time-series processing

Input: Measurements over time

System: Updates state as each value arrives

Output: A forecast or anomaly score

Speech and sequence tasks

Input: Ordered audio frames

System: Tracks useful patterns across frames

Output: A label, transcription, or signal estimate

What it is NOT

RNN ≠ LSTM

RNN: The broader recurrent-network family.

LSTM: A recurrent architecture with gates for handling longer-range information.

RNN ≠ Transformer

RNN: Processes a sequence recurrently, step by step.

Transformer: Uses attention to relate positions in a sequence.

LSTM ≠ Long-term Human Memory

LSTM: A mathematical mechanism for retaining useful signals.

Human memory: A biological and cognitive process.

Remember this

RNNs process sequences recurrently, while LSTMs add mechanisms that help preserve useful information over longer ranges.

Video

no video asset available

A video explainer will be added when a real source is available.