class: center, middle # Ch0: Introduction to Deep Learning ### Clément Dombry .affiliations[    ] .credits[ Based on the lecture notes and slides by Charles Ollion et Olivier Grisel [available on Github](https://github.com/m2dsupsdlclass/lectures-labs) <br> THANKS TO THEM ! ] --- # Goal of the class ## Understanding DL - When and where to use DL ? - Understand the most important models (MLP, CNN) - Understand the most important concepts (training, regularisation, transfer learning) -- ## Practice of DL - Implement using `Keras/Tensorflow` - Engineering for building and training DL --- # Short history: early ages - Back from 1950, the artificial intelligence community is trying to mimick the mechanisms in the brain with artificial neurons and neural networks. .center[ <img src="images/neural_net_bio.png" style="width: 670px;" /> ] - F. Rosenblatt (1958). The perceptron: a probabilistic model for information storage and organisation in the brain. - P. Werbos's PhD (1974) on the training of multilayer perceptron with backpropagation algorithm. - Allthough the theory was avalaible, practice was difficult and training of deep neural networks was not possible. --- # Short history: take-off - First successes in the 90’s, with applications in image and speech recognition (Yan LeCun). - 2009: Nvidia Graphical Processing Units (GPU) were used to train deep neural networks, speeding up training of a factor 100 (from weeks to hours or days). - 2012: ImageNet challenge - CNN outperformed current methods for large scale image classification.<br> <small> Error rate: 25% in 2011, 16% in 2012, 4% in 2016. </small> - 2015: AlphaGo developped by Deepmind wins against the European champion, in 2017 against the world champion. - 2015: TensorFlow, free and open-source software for DL, was released under apache license by Google. In 2019, TensorFLow2/Keras provides a simple API making DL accessible. --- # What is Deep Learning ? .center[ <img src="images/image_dl.png" style="width: 700px;" /> ] - Good old Neural Networks, with more layers/modules - Non-linear, hierarchical, abstract representations of data - Flexible models with any input/output type and size - Differentiable Functional Programming (Tensorflow) --- # Why Deep Learning Now? - Better algorithms & understanding - .grey[Computing power (GPUs, TPUs, ...)] - .grey[Data with labels] - .grey[Open source tools and models] .center[ <img src="images/Goodfellow.jpg" style="height: 200px;" /> <img src="images/Chollet.jpg" style="height: 200px;" /> <img src="images/Geron.jpg" style="height: 200px;" /> ] --- # Why Deep Learning Now? - Better algorithms & understanding - Computing power (GPUs, TPUs, ...) - .grey[Data with labels] - .grey[Open source tools and models] .center[ <img src="images/gpu_tpu.png" style="width: 450px;" /><br/><br/> <small>_GPU and TPU_</small> ] --- # Why Deep Learning Now? - Better algorithms & understanding - Computing power (GPUs, TPUs, ...) - Data with labels - .grey[Open source tools and models] .center[ <img src="images/ng_data_perf.svg" style="width: 400px;" /><br/><br/> <small>_Adapted from Andrew Ng_</small> ] --- # Why Deep Learning Now? - Better algorithms & understanding - Computing power (GPUs, TPUs, ...) - Data with labels - Open source tools and models .center[ <img src="images/frameworks.png" style="width: 500px;" /><br/><br/> ] --- # DL Today: Speech-to-Text .center[ <img src="images/speech.png" style="width: 780px;" /> ] --- # DL Today: Vision .center[ <img src="images/vision.png" style="width: 720px;" /> ] --- # DL Today: Vision .center[ <img src="images/vision2.png" style="width: 720px;" /> ] --- # DL Today: NLP .center[ <img src="images/nlp.png" style="width: 600px;" /> ] --- # DL Today: NLP .center[ <img src="images/nlp2.png" style="width: 720px;" /> ] -- Most of chatbots claiming "AI" do not use Deep Learning (yet?) --- # DL Today: Vision + NLP .center[ <img src="images/nlp_vision.png" style="width: 760px;" /> ] --- # DL Today: Image translation .center[ <img src="images/vision_translation.png" style="width: 700px;" /> ] --- # DL Today: Generative models .center[ <img src="images/nvidia_celeb.jpg" style="width: 350px;" /> <br/>Sampled celebrities [Nvidia 2017] ] -- <br/> .center[ <img src="images/stackgan.jpg" style="width: 600px;" /> <br/>StackGAN v2 [Zhang 2017] ] --- # DL Today: Generative models .center[ <img src="images/WaveNet.gif" style="width: 400px;" /> <br/>Sound generation with WaveNet [DeepMind 2017] ] -- <br/> Guess which one is generated? .center[ <audio controls><source src="images/columbia_gen.wav"></audio> <br/> <audio controls><source src="images/columbia_gt.wav"></audio> <small>_Tacotron 2 Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions, 2017_</small> ] --- # Language / Image models Open-AI GPT-3, or DALL-E: https://openai.com/blog/dall-e/ .center[ <img src="images/dalle.png" style="width: 600px;" /> ] --- # DL in Science: Genomics .center[ <img src="images/deepgenomics.png" style="width: 580px;" /> ] -- .center[ <img src="images/protein_fold.gif" style="width: 320px;" /><br/> <small>[AlphaFold by DeepMind](https://deepmind.com/blog/article/alphafold-a-solution-to-a-50-year-old-grand-challenge-in-biology)</small> ] --- # DL in Science: Chemistry, Physics .center[ <img src="images/deep_other.png" style="width: 680px;" /> ] --- # DL in Science: Chemistry, Physics .center[ <img src="images/Accelerating_Eulerian_Fluid_Simulation_with_Convolutional_Networks.gif" style="width: 350px;" /> ] - Finite element simulator accelerated (~100 fold) by a 3D convolutional network --- # DL for AI in games .center[ <img src="images/games.png" style="width: 650px;" /> ] -- <small> AlphaGo/Zero: Monte Carlo Tree Search, Deep Reinforcement Learning, self-play </small> --- # Outline of the class ## Course - Ch1: Linear regression and logistic regression revisited - Ch2: MLP (MultiLayer Perceptron) and backpropagation - Ch3: CNN (Convolution Neural Networks) and image classification - Ch4: Optimization and Regularization ## Labs - Mostly using 'Keras/Tensorflow' --- # Organisation and evaluation ## Organisation - 4x3h = 12h course (with Clément Dombry) - 4x3h = 12h lab (with Romain Pic) -- ## Evaluation - exam in January (50% of final mark) - project with defense in January (50%) --- # Beyond this course ## Recommended books - [Ian Goodfellow's book](http://www.deeplearningbook.org/): .small[math and main concepts] - [Francois Chollet's book](https://www.manning.com/books/deep-learning-with-python): .small[Keras programming] - [Aurélien Géron's book](https://www.oreilly.com/library/view/hands-on-machine-learning/9781492032632/): .small[Machine Learning with Scikit-learn and Deep Learning with TF/Keras] -- ## Recommended online courses - [Institut Polytechnique](https://github.com/m2dsupsdlclass/lectures-labs): .small[Master course on deep learning] - [Coursera course from Imperial College London](https://www.coursera.org/learn/getting-started-with-tensor-flow2): .small[Getting started with TensorFlow 2, Customizing your Models with TensorFlow2, Probabilistic Deep Learning with Tensorflow2]