Descriptif

"Deep Learning I"
M2 Data-Science
Georoy Peeters, Alasdair Newson (Telecom Paris)
2022-2023

Teachers: Georoy Peeters, Alasdair Newson (Telecom Paris, IP-Paris)

Deep Learning (machine learning based on deep artificial neural networks) has become extremely popular over the last years due to the very good results it allows for tasks such as regression, classification or genera- tion. The objective of this course is to provide a theoretical understanding and a practical usage of the three main types of networks (Multi-Layer- Perceptron, Recurrent-Neural-Network and Convolutional Neural Network). The content of this course ranges from the perceptron to the generation of adversarial images. Each theoretical lecture is followed by a practical lab on the corresponding content where student learn to implement these networks using the currently three popular frameworks: pytorch, tensorflow and keras.

Format : 6 sessions of 3.5 hours + Exam

Lectures content:

• Multi-Layer-Perceptron (MLP): Perceptron, Logistic Regression, Chain rule, Back-propagation, Deep Neural Activation functions, Vanishing gradient, Ini- tialization, Regularization (L1,L2,DropOut), Alternative Gradient Descent, Batch-

            normalization

• Recurrent Neural Network (RNN) : Simple RNN, Forward Propagation, Backward Propagation Through Time, Vanishing/ Exploding gradients, Gated Units (LSTM, GRU), Various architectures, Sequence-to-sequence, Attention model
• Convolutional Neural Network (CNN) : CNNs use sparse connectivity and weight sharing to reduce parameters and create more powerful networks , connections are organized in a convolution operation, CNNs now provide the state- of-the-art in a vast array of problems, we will see how CNNs work and we will implement them for classification problems

Labs content :

• text recognition, sentiment classification 
• music generation
• image recognition
• image generation

Programming language

• Python (numpy, scikit-learn, matplotlib)
• DL frameworks: pytorch, tensorflow, keras
• Use Télécom computers, your own labtop or colab.research.google.com → needs a Google account → open one before the first Lab !
• a Graphics Processing Unit (GPU) will not be required, however if you have one, this will speed up the learning process

Grading : 30% labs/project + 70% written exam