Descriptif

The main purpose of this course is to introduce the mathematical formalism of the learning theory and to showcase its relations with more classical statistical theory of nonparametric estimation. During the lectures, some exercises will be given and solved.

• Presentation of 3 central problems: regression, binary classification, clustering or density estimation. Connection between these problems.

• Universal consistency. Overfitting and underfitting. The Hoeffding inequality and empirical risk minimisation. Rademacher complexities.

• Density estimation by histograms. Bias-variance decomposition and the rate of convergence over Holder classes.

• Adaptive choice of the bandwidth by the method of estimated unbiased risk minimization. Local choice of the bandwidth by the Lepski method.

• Nonparametric regression and sparsity. Thresholding Fourier coefficients.

 Evaluation : final exam, 2h.