← Deep Learning learning pathDeep Learning · Module 7 of 10
Sequence models and attention
Handle text, audio and time series with RNNs and Transformers.
📖 Key ideas
- RNNs process sequences step by step with memory.
- LSTMs/GRUs reduce forgetting over long sequences.
- Attention lets models focus on relevant parts of input.
- Transformers use self-attention and process sequences in parallel.
- Transformers power LLMs like GPT and Claude.
💡 Real-life example: Attention is a student highlighting the most relevant lines in a textbook before answering.
RNNRecurrent Neural Network
LSTMLong Short-Term Memory
Self-attentionEach token attends to others
TransformerAttention-based architecture