The objective of this course is to introduce the theory of linear and non-linear regression. Regression is a simple yet versatile model used in many problems both in applications and in fundamental research. It is therefore absolutely essential for a data scientist to understand the theory of regression analysis. As such, this course will be theory oriented. Practical aspects will also be treated in tutorials.
The first part of the course will be devoted to the theory of linear models. The theoretical analysis of the statistical properties of this model relies on an elegant combination of basic Linear Algebra and Probability theory. The main goal of this part will be to develop an in-depth understanding of statistical inference in the linear model relying only on the geometric structure of the model.
In the second part of the course, we will explore important topics about regression: Model selection, penalized empirical risk regularization, high-dimensional statistics and non-parametric statistics. Time permitting, we will also investigate generalized linear models such as logistic regression.
Part I: Linear Models
- Gaussian Vectors
- Random vectors
- Gaussian Vectors
- Cochran Theorem
- Linear functionals
- Gauss-Markov optimality
- Estimation of mean and variance
- Optimality of MLE
- James-Stein Phenomenon
- LRT and Fisher Test
- CI for linear functionals of mean
Part II: Advanced Topics
- Model Selection
- Penalized Empirical Risk minimization : AIC, BIC
- Case of orthogonal design
- Practical model selection
- Sparsity oracle inequalities
- LASSO Estimator
- Variable Selection
- Kernel Estimator
- Pointwise error of estimation
- Minimax rate of estimation
- Tuning the bandwidth parameter
- Course evaluation : Final Exam
- Course Language: Either in English or in French with lecture material in English
Format des notesNumérique sur 20Littérale/grade réduit
Pour les étudiants du diplôme Diplôme d'ingénieur de l'Ecole polytechniqueLe rattrapage est autorisé
- Crédits ECTS acquis : 5 ECTS
La note obtenue rentre dans le calcul de votre GPA.