← Deep Learning learning pathDeep Learning · Module 3 of 10
Activation and loss functions
Choose activations and measure errors with loss functions.
📖 Key ideas
- ReLU: max(0, x), the default for hidden layers.
- Sigmoid outputs 0–1 (binary probability); softmax outputs class probabilities.
- Loss measures how wrong predictions are.
- MSE for regression; cross-entropy for classification.
- Training tries to minimise the loss.
💡 Real-life example: The loss function is a teacher's red pen: it marks how far each answer is from correct.
ReLURectified Linear Unit
SoftmaxTurns scores into probabilities
Cross-entropyLoss for classification
MSEMean squared error