Madhavan Mukund

Advanced Machine Learning

Sep–Dec, 2021

Administrative details

• Instructors: Pranabendu Misra, Madhavan Mukund

• Lectures and Tutorials: Two live online classes each week. Tutorials as needed.

• Evaluation:

  • Quizzes on Moodle, midsemester exam, programming assignments, final exam

  • Weightage approximately 10%, 20%, 30%, 40%, to be confirmed

  • Copying is fatal

• Text and reference books:

  • Avrim Blum, John Hopcroft and Ravi Kannan: Foundations of Data Science, Cambridge University Press 2021 Available Here

  • Ian Goodfellow, Yoshua Bengio and Aaron Courville: Deep Learning, MIT Press 2016 Available Here

  • Richard S. Sutton and Andrew G. Barto: Reinforcement Learning: An Introduction, MIT Press (2nd ed) 2018 Available Here

  • Aurélien Géron: Hands-On Machine Learning with Scikit-Learn, Keras and Tensorflow (2nd ed), O'Reilly 2019

  • Francois Chollet: Deep Learning with Python, Manning Publications 2017

  • Nikhil Buduma: Fundamentals of Deep Learning, O'Reilly 2017

  • Stuart J Russell and Peter Norvig: Artificial Intelligence: A Modern Approach, Pearson (3rd ed) 2016

  • Daphne Koller and Nir Friedman: Probabilistic Graphical Models – Principles and Techniques, MIT Press 2009

Course plan

This list is approximate and subject to change.

• Deep Learning Philosophy, Deep Neural Networks, Convolutional Neural Networks, Recurrent Neural Networks, LSTM, Autoencoders, Generative Adversarial Networks
• Gaussian Process Regression
• Bayesian optimization
• Reinforcement Learning
• Probabilistic Graphical Models and Causality

Assignments

• Assignment 1, 31 Oct 2021, due 14 Nov 19 Nov 2021

• Assignment 2, 5 Dec 2021, due 19 Dec 2021

• Assignment 3, 20 Dec 2021, due 2 Jan 7 Jan 2022

Lectures

• Lecture 1: 20 Sep 2021
  (Introduction (pdf), Theoretical foundations of ML (pdf), Class Notes (pdf), Lecture video)

  PAC learning, uniform convergence

  Representational capacity, overfitting, underfitting, hyperparameter search

  • Blum et al, Chapter 5.1, 5.4, 5.4.1
  • Goodfellow et al, Chapter 5.2

• Lecture 2: 23 Sep 2021
  (VC-Dimension (pdf), Loss Functions (pdf), Class Notes (pdf), Lecture video)

  VC dimension

  • Blum, Hopcroft and Kannan, Chapter 5.5, upto 5.5.2

  Choosing loss functions: MLE, cross entropy

  • Goodfellow et al, Chapter 5.5, 3.13

• Lecture 3: 28 Sep 2021
  (Training Deep Neural Networks (pdf), Class Notes (pdf), Lecture video)

  Neural networks, unstable gradients, initialization strategies, non-saturating activation functions, batch normalization

  • Géron, Chapter 11

• Lecture 4: 30 Sep 2021
  (Class Notes (pdf), Lecture video)

  ill-conditioning; optimizing backpropagation: momentum, adaptive learning rates; regularization; local minima; model identifiability; saddle points

  • Buduma, Chapter 4
  • Géron, Chapter 11

• Lecture 5: 04 Oct 2021
  (DNN for MNIST (html), DNN for MNIST (ipynb), Lecture video – Part 1, Part 2)

  Deep neural network for MNIST

• Lecture 6: 07 Oct 2021
  (Convolutional Neural Networks (pdf), Lecture video)

  Convolution, feature maps, zero padding, stride, pooling, parameter sharing

  • Géron, Chapter 14
  • Buduma, Chapter 5
  • Goodfellow et al, Chapter 9, 9.1, 9.2, 9.3, 9.5

• Lecture 7: 18 Oct 2021
  (Convolutional Neural Networks (zip), Lecture video)

  CNNs, adversarial attacks

  • Dive into Deep Learning, Chapter 6
  • Deep Learning: ConvNet Evolutions, Architectures, Implementation Details and Advantages

• Lecture 8: 21 Oct 2021
  (Recurrent Neural Networks (pdf), Lecture video)

  RNNs

  • Géron: Chapter 15
  • A Critical Review of Recurrent Neural Networks for Sequence Learning, Zachary C. Lipton, John Berkowitz, Charles Elkan
  • Understanding LSTM Networks

• Lecture 9: 25 Oct 2021
  (RNN and LSTM examples (html), (RNN and LSTM examples (ipynb), Lecture video)

  RNN and LSTM examples

• Lecture 10: 28 Oct 2021
  (Autoencoders (pdf), Lecture video)

  Autoencoders: standard, de-noising, contractive, variational

  • Introduction to autoencoders, Alfredo Canziani
  • Generative Models – Variational Autoencoders, Alfredo Canziani
  • Variational Autoencoders are Beautiful, Steven Flores
  • Chapters 14 and 20.10.3, Goodfellow et.al.

• Lecture 11: 01 Nov 2021
  (Autoencoders (zip), Lecture video)

  Autoencoders, denoising autoencoders, variational autoencoders

• Lecture 12: 15 Nov 2021
  (Bayesian Optimization (pdf), Lecture video)

  Bayesian Optimization, Gaussian Process Regression

  • A Tutorial on Bayesian Optimization …, Brochu et al
  • Gaussian Processes for Regression: A Quick Introduction, M. Ebden
  • A Tutorial on Bayesian Optimization, Peter I. Frazier
  • Bayesian optimization with skopt

• Lecture 13: 18 Nov 2021
  (Generative Adversarial Networks (pdf), Lecture video)

  Generative Adversarial Networks (GANs)

  • Deep Generative Models, Ava Soleimany (pdf, video)

• Lecture 14: 22 Nov 2021
  (Generative Adversarial Networks (pdf), Lecture video)

  • GAN to generate celebrity faces (html, ipynb)

• Lecture 15: 25 Nov 2021
  (Reinforcement Learning (pdf), Class notes (pdf), Lecture video)

  Introduction to reinforcement learning, multi-armed bandits

  • Sutton and Barto, Chapters 1, 2.1–2.7
  • Russell and Norvig, Chapter 17.1

• Lecture 16: 29 Nov 2021
  (Markov Decision Processes (pdf), Class notes (pdf), Lecture video)

  Markov Decision Processes: Basic definitions and examples, policies and value functions, Bellman equation, optimal policies

  • Sutton and Barto, Chapter 3

• Lecture 17: 02 Dec 2021
  (MDPs: Dynamic Programming (pdf), Class notes (pdf), Lecture video)

  Markov Decision Processes: Police evaluation, policy iteration, value iteration

  • Sutton and Barto, Chapter 4

• Lecture 18: 06 Dec 2021
  (MDPs: Dynamic Programming (pdf), Class notes (pdf), Lecture video)

  Markov Decision Processes: Monte Carlo methods

  • Sutton and Barto, Chapter 5

• Lecture 19: 09 Dec 2021
  (MDPs: Temporal Difference Learning (pdf), Class notes (pdf), Lecture video)

  Markov Decision Processes: Temporal Difference Learning

  • Sutton and Barto, Chapter 6

• Lecture 20: 13 Oct 2021
  (Deep Reinforcement Learning (pdf), DQN for Video Games (html), DQN for Video Games (ipynb), Lecture video)

  Deep reinforcement learning

  • Reinforcement Learning, lecture by Fei-Fei Li, Justin Johnson and Serena Yeung (Slides, Video)
  • Playing Atari with Deep Reinforcement Learning, Mnih et al

• Lecture 21: 16 Dec 2021
  (Probabilistic Graphical Models (pdf), Class notes (pdf), Lecture video)

  Probabilistic Graphical Models: Inference, Conditional Independence

  • Russell and Norvig, Chapter 14.1, 14.2.1, 14.2.2, 14.4.1
  • Probabilistic Graphical Models: Principles and Techniques by Daphne Koller and Nir Friedman, Chapter 3.2, 3.3
  • Notes on D-separation by Richard Scheines