Descriptif
Brief presentation. The goal of this course is to present the two main modern tools in
probability and essential objects from a theoretical perspective as well as in applications:
martingales and Markov chains. Both pertain to the theory of stochastic processes in
discrete time, namely sequences of random variables which are not independent, but
rather in which the law at a given time depends on the past.
Martingale theory constitutes a fantastic tool that for example allows to describe the
law of the time and position of the first entry of such a process in a given subset as well
as to establish almost sure convergence as time tends to infinity. Markov chains appear
very naturally in the modelisation of various phenomena for it describes the evolution of
a stochastic process in which at a given time, the law of the next position in fact only
depends on the present position and not the whole past trajectory.
Prerequisites. This course is meant to be a second course on probability theory. Famili-
arity with basic measure theory and probability such as random variables, their law and
expectation, independence, Lp spaces, different notion of convergences, Law of Large
Numbers & Central Limit Theorem will be assumed. Some of these notions will be briefly
recapped and are covered in details in some of the references below if needed. Some
familiarity with Python for numerical applications can be useful.
Practical informations. The course consists of 2h of lectures and 2h of exercises ses-
sions per week, leaving plenty of time to work on your own using e.g. the references
below. Preparing the exercise in advance is necessary for the sessions to be useful. The
exercise sessions will also discuss simulations in Python.
The grade will be based on two written exams, one in the middle of the course and a
final one.
Diplôme(s) concerné(s)
Parcours de rattachement
Format des notes
Numérique sur 20Littérale/grade réduitPour les étudiants du diplôme M1 Applied Mathematics and statistics
Le rattrapage est autorisé (Note de rattrapage conservée)- Crédits ECTS acquis : 7.5 ECTS
Programme détaillé
1
• Conditional expectation and generalities on discrete-time stochastic processes: fil-
trations, stopping times, Kolmogorov extension theorem;
• Martingales: maximal inequality, stopping theorem, convergence theorems, uni-
form integrability, applications;
• Markov chains: transition kernel, the (strong) Markov property, invariant measure,
recurrence/transience, convergence theorems, applications.
References. Here are some books that can be useful in relation with this course. Feel
free to look outside this list, the important point is to find one or more that you enjoy
reading and find complementary to the lectures.
• The following references cover the basics of measure theory and probability, as well
as the conditional expectation and martingale part of this course; only the first one
also treats Markov chains. They offer a complete course with also many exercises
and examples.
– Rick Durrett. Probability: Theory and Examples. Available in pdf at https:
//services.math.duke.edu/~rtd/PTE/pte.html.
– Geoffrey Grimmett and David Stirzaker. Probability and Random Processes.
– Allan Gut. Probability A Graduate Course.
– Jean Jacod and Philip Protter. Probability Essentials.
– David Williams. Probability with Martingales.
• The following references only contain a short recap of the definitions and main
results (without proof) or not at all, but are a great source of solved exercises.
– Paolo Baldi, Laurent Mazliak, and Pierre Priouret. Martingales and Markov
Chains - Solved Exercises and Elements of Theory.
– Geoffrey Grimmett and David Stirzaker. One Thousand Exercises in Probabil-
ity.
• Finally, the last two references are entirely dedicated to Markov chains and extend
way beyond the scope of this course.
– Randal Douc, Eric Moulines, Pierre Priouret, and Philippe Soulier. Markov
Chains.
– David Levin, Yuval Peres, and Elizabeth Wilmer. Markov Chains and Mixing Times.
