Reinforcement Learning for Large Language Models: A Modern View is a 3-hour tutorial for the Toronto Machine Learning Summit. It provides a motivation-first, mathematically rigorous introduction to reinforcement-learning-style post-training for large language models, aimed at machine learning researchers and advanced students who want a principled view of the methods behind modern LLM alignment and adaptation.

The tutorial starts with a brief overview of the contemporary LLM post-training pipeline and then develops the policy-gradient foundations needed to understand these methods from first principles. Instead of treating the field as a sequence of named algorithms, it organizes the material around the major design dimensions that distinguish practical approaches: how the training signal is obtained, how variance is reduced, how policy drift is controlled, how KL regularization is imposed and estimated, and how credit is assigned within a completion. Throughout, the tutorial emphasizes the tradeoffs encoded by these choices and distinguishes clearly between mathematically established results, theory-motivated arguments, practical heuristics, and empirical findings.

This perspective is then used to place methods such as REINFORCE, PPO-style RLHF, DPO, RLOO, and GRPO in a common mathematical framework, and to connect them to published descriptions of post-training in recent frontier models. Participants will leave with a unified understanding of the foundations of LLM post-training, the main ideas behind current methods, and the research questions that remain open.