Internship← Generative AI & Prompt Engineering learning pathGenerative AI & Prompt Engineering · Module 2 of 10
How LLMs work
Understand tokens, training and limitations.
📖 Key ideas
- LLMs predict the next token given previous tokens.
- Pretraining on huge text, then fine-tuning and alignment (RLHF).
- Temperature controls randomness.
- Knowledge has a cutoff date; models may not know recent events.
- Bias and errors can come from training data.
💡 Real-life example: An LLM is autocomplete that read a giant library; it predicts words, not truth.
Next-token predictionCore LLM task
PretrainingLearning from large text corpora
RLHFReinforcement Learning from Human Feedback
TemperatureRandomness setting