Descriptif

This course provides a theoretically grounded introduction to reinforcement learning (RL). The course focuses on the workhorse of modern continuous RL: policy gradient methods. It begins with the foundations of Markov decision processes (MDPs) and dynamic programming, followed by an introduction to policy gradient methods. The course then proceeds to derive improvements to this algorithm by covering policy evaluation and baselines, policy optimization and actor-critic methods, and the descent lemma that leads to modern methods such as NPG and PPO. An emphasis is given on the underlying mathematical constructions to understand the different algorithm heuristics. This course includes a special session on the use of RL for LLM post-training.