Archives

A from-fundamentals tour of concurrency in Python, told as one connected story instead of a glossary of terms — threads vs. processes, the GIL, locks, condition variables, asyncio, distributed rate limiting, and a staged concurrent web crawler exercise.

How DDP, tensor/pipeline/sequence parallelism, ZeRO/FSDP, activation recomputation, mixed precision, and the optimizer stack combine to train models no single GPU could hold.

The encoder-decoder architecture that powered translation before Transformers, the fixed-size bottleneck it created, and the idea that finally fixed it.

The architectures that came before Transformers, the limits they hit, and why each new design was needed to fix the previous one.

Why models fail in two opposite ways — being too rigid or too sensitive — and how to find the sweet spot between them.

What overfitting is, why it happens, and how dropout stops a network from memorising the training data.

A deep dive into the transformer architecture, why decoder-only models won, and the key design decisions — RoPE, GQA, Flash Attention, MoE — that define every modern LLM.

Why activations drift as they pass through deep networks, and how BatchNorm and LayerNorm fix it in different ways.

Why plain gradient descent isn't enough, and how SGD, momentum, Adam, and AdamW each fix a problem the previous one had.

How gradient descent uses the loss to update weights, and how backpropagation computes the gradients that make it possible.

What loss functions are, how MSE and cross-entropy work, and why picking the wrong one breaks your model even if everything else is right.

Why stacking linear layers isn't enough, and how activation functions like ReLU, GELU, and SiLU give neural networks their power.

A neural network explained from scratch - neurons, weights, layers, and the forward pass - no ML background required.

Full table of contents for the GenZ to AI Enz series - every post and walkthrough in order.

A complete series taking CS students and early-career engineers from zero ML knowledge to building real AI systems with LLMs and agents.

Fine-tuning Microsoft's Phi-2 using Direct Preference Optimization (DPO) on the Anthropic Helpful and Harmless dataset with LoRA and 8-bit quantization.

The architecture behind our async model serving platform at Instabase — async workers, RabbitMQ, multi-level caching, and sticky routing to cut inference time by 40%.

Our EMNLP 2021 paper on abstractive summarization of Hindi-English code-switched conversations — introducing the GupShup dataset.