Tag: ml-engineering
All the articles with the tag "ml-engineering".
-
Dropout and Overfitting: Teaching a Network Not to Cheat
What overfitting is, why it happens, and how dropout stops a network from memorising the training data.
-
Normalization: BatchNorm, LayerNorm, and Why Transformers Need a Different One
Why activations drift as they pass through deep networks, and how BatchNorm and LayerNorm fix it in different ways.
-
Optimizers: SGD, Momentum, Adam, and AdamW
Why plain gradient descent isn't enough, and how SGD, momentum, Adam, and AdamW each fix a problem the previous one had.
-
Gradient Descent and Backpropagation: How a Network Actually Learns
How gradient descent uses the loss to update weights, and how backpropagation computes the gradients that make it possible.