Tag: ml-engineering
All the articles with the tag "ml-engineering".
-
Distributed Training at Scale: Parallelism, Memory, and the Optimizer Stack
How DDP, tensor/pipeline/sequence parallelism, ZeRO/FSDP, activation recomputation, mixed precision, and the optimizer stack combine to train models no single GPU could hold.
-
Seq2Seq: Teaching Machines to Translate
The encoder-decoder architecture that powered translation before Transformers, the fixed-size bottleneck it created, and the idea that finally fixed it.
-
CNN, RNN, LSTM: The Road to Transformers
The architectures that came before Transformers, the limits they hit, and why each new design was needed to fix the previous one.
-
Bias, Variance, and the Tradeoff Every Model Faces
Why models fail in two opposite ways — being too rigid or too sensitive — and how to find the sweet spot between them.