GenAI Interview Prep

A deep-dive series for experienced ML engineers preparing for GenAI and LLM engineering roles — from transformer internals to production inference, RAG, agents, and system design.

• Transformer Architecture & Key Design Decisions

  6 May, 2026

  A deep dive into the transformer architecture, why decoder-only models won, and the key design decisions — RoPE, GQA, Flash Attention, MoE — that define every modern LLM.

• Distributed Training at Scale: Parallelism, Memory, and the Optimizer Stack

  17 Jul, 2026

  How DDP, tensor/pipeline/sequence parallelism, ZeRO/FSDP, activation recomputation, mixed precision, and the optimizer stack combine to train models no single GPU could hold.