Descriptif
Objectives
The objective of this course is to introduce the theory of linear and non-linear regression. Regression is a simple yet versatile model used in many problems both in applications and in fundamental research. It is therefore absolutely essential for a data scientist to understand the theory of regression analysis. As such, this course will be theory oriented. Practical aspects will also be treated in tutorials.
The first part of the course will be devoted to the theory of linear models. The theoretical analysis of the statistical properties of this model relies on an elegant combination of basic Linear Algebra and Probability theory. The main goal of this part will be to develop an in-depth understanding of statistical inference in the linear model relying only on the geometric structure of the model.
In the second part of the course, we will explore important topics about regression: Model selection, penalized empirical risk regularization, high-dimensional statistics and non-parametric statistics. Time permitting, we will also investigate generalized linear models such as logistic regression.
Syllabus
Part I: Linear Models
- Gaussian Vectors
- Random vectors
- Gaussian Vectors
- Cochran Theorem
- Gauss-Markov Estimation
- Linear functionals
- Gauss-Markov optimality
- Estimation of mean and variance
- Normal Estimation
- MLE
- Optimality of MLE
- James-Stein Phenomenon
- Statistical Testing and Confidence Intervals
- LRT and Fisher Test
- CI for linear functionals of mean
- Anova/Ancova
Part II: Advanced Topics
- Model Selection
- Penalized Empirical Risk minimization : AIC, BIC
- Case of orthogonal design
- Practical model selection
- High-dimensional Statistics
- Sparsity oracle inequalities
- LASSO Estimator
- Variable Selection
- Nonparametric regression
- Kernel Estimator
- Pointwise error of estimation
- Minimax rate of estimation
- Tuning the bandwidth parameter
- Logistic Regression
- Course evaluation : Final Exam
- Course Language: Either in English or in French with lecture material in English
effectifs minimal / maximal:
/70Diplôme(s) concerné(s)
- Echanges PEI
- Non Diplomant
- M1 Mathématiques et Applications - Voie Jacques Hadamard - École Polytechnique
- Titre d’Ingénieur diplômé de l’École polytechnique
- M1 Data AI - Data and Artificial Intelligence
Parcours de rattachement
Format des notes
Numérique sur 20Littérale/grade réduitPour les étudiants du diplôme M1 Mathématiques et Applications - Voie Jacques Hadamard - École Polytechnique
L'UE est acquise si note finale transposée >=- Crédits ECTS acquis : 5 ECTS
Pour les étudiants du diplôme Non Diplomant
L'UE est acquise si note finale transposée >=- Crédits ECTS acquis : 5 ECTS
Pour les étudiants du diplôme M1 Data AI - Data and Artificial Intelligence
Le rattrapage est autorisé (Note de rattrapage conservée)- Crédits ECTS acquis : 5 ECTS
Pour les étudiants du diplôme Titre d’Ingénieur diplômé de l’École polytechnique
Le rattrapage est autorisé (Note de rattrapage conservée)- Crédits ECTS acquis : 5 ECTS
La note obtenue rentre dans le calcul de votre GPA.
Pour les étudiants du diplôme Echanges PEI
Le rattrapage est autorisé (Note de rattrapage conservée)- Crédits ECTS acquis : 5 ECTS