v2.11.0 (5518)

PA - C1'' - MAP553 : The Art of regression

Domaine > Mathématiques appliquées.



The objective of this course is to introduce the theory of linear and non-linear regression. Regression is a simple yet versatile model used in many problems both in applications and in fundamental research. It is therefore absolutely essential for a data scientist to understand the theory of regression analysis. As such, this course will be theory oriented. Practical aspects will also be treated in tutorials. 

The first part of the course will be devoted to the theory of linear models. The theoretical analysis of the statistical properties of this model relies on an elegant combination of basic Linear Algebra and Probability theory. The main goal of this part will be to develop an in-depth understanding of statistical inference in the linear model relying only on the geometric structure of the model.

In the second part of the course, we will explore important topics about regression: Model selection, penalized empirical risk regularization, high-dimensional statistics and non-parametric statistics. Time permitting, we will also investigate generalized linear models such as logistic regression.


Part I: Linear Models

  1. Gaussian Vectors
    • Random vectors
    • Gaussian Vectors
    • Cochran Theorem
  2. Gauss-Markov Estimation
    • Linear functionals
    • Gauss-Markov optimality 
    • Estimation of mean and variance
  3. Normal Estimation
    • MLE
    • Optimality of MLE
    • James-Stein Phenomenon
  4. Statistical Testing and Confidence Intervals
    • LRT and Fisher Test
    • CI for linear functionals of mean
  5. Anova/Ancova


Part II: Advanced Topics

  1. Model Selection
    • Penalized Empirical Risk minimization : AIC, BIC
    • Case of orthogonal design
    • Practical model selection
  2. High-dimensional Statistics
    • Sparsity oracle inequalities
    • LASSO Estimator
    • Variable Selection
  3. Nonparametric regression
    • Kernel Estimator
    • Pointwise error of estimation
    • Minimax rate of estimation
    • Tuning the bandwidth parameter
  4. Logistic Regression
  • Course evaluation : Final Exam
  • Course Language: Either in English or in French with lecture material  in English


Format des notes

Numérique sur 20

Littérale/grade réduit

Pour les étudiants du diplôme M1 Informatique - Voie Jacques Herbrand - X

Le rattrapage est autorisé (Note de rattrapage conservée)

    Pour les étudiants du diplôme Diplôme d'ingénieur de l'Ecole polytechnique

    Le rattrapage est autorisé (Note de rattrapage conservée)
      L'UE est acquise si note finale transposée >= C
      • Crédits ECTS acquis : 5 ECTS

      La note obtenue rentre dans le calcul de votre GPA.

      Pour les étudiants du diplôme Echanges PEI

      Le rattrapage est autorisé (Note de rattrapage conservée)
        L'UE est acquise si note finale transposée >= C
        • Crédits ECTS acquis : 5 ECTS
        Veuillez patienter