Madhavan Mukund

Advanced Machine Learning

Aug-Nov, 2019

Administrative details

• Instructors: Sourish Das, Madhavan Mukund

• Evaluation:

  • Assignments TBA, midsemester exam TBA, final exam TBA

  • Copying is fatal

• Text and reference books:

  • Ian Goodfellow, Yoshua Bengio and Aaron Courville: Deep Learning, MIT Press 2016 Available Here
  • Richard S. Sutton and Andrew G. Barto: Reinforcement Learning: An Introduction, MIT Press (2nd ed) 2018 Available Here
  • Francois Chollet: Deep Learning with Python, Manning Publications 2017
  • Stuart J Russell and Peter Norvig: Artificial Intelligence: A Modern Approach, Pearson (3rd ed) 2016
  • Daphne Koller and Nir Friedman: Probabilistic Graphical Models – Principles and Techniques, MIT Press 2009

Course plan

(To be expanded and updated. Highly subject to change.)

• Deep Learning Philosophy, Deep Neural Networks, Convolutional Neural Networks, Recurrent Neural Networks, LSTM
• Pytorch, Keras
• Reinforcement Learning
• Graphical Network Models
• Hidden Markov Models

Assignments

• Assignment 1, 22 Oct 2019, due 3 Nov 2019

• Assignment 2, 15 Nov 2019, due 30 Nov 2019

Lecture summary

• Lecture 1, 13 Aug 2019: Class notes

  Overview of PAC learning, capacity, underfitting, overfitting, bias, variance

  • Goodfellow et al, Chapter 5.1, 5.2, 5.3, 5.4 (part), 5.9 (part), 5.10 (part)

• Lecture 2, 20 Aug 2019: Class notes

  Feed-forward networks: nonlinearity, XOR example, cost functions, MLE and cross-entropy

  • Goodfellow et al, Chapter 6, 6.1, 6.2.1.1, 3.1.3

• Lecture 3, 22 Aug 2019: Class notes

  Feed-forward networks: output layer activation functions; hidden layer activation functions; architecture; backpropagation and stochastic gradient descent; optimization – momentum; adaptive learning rates – delta-bar-delta, AdaGrad, RMSprop, Adam; initializing neural networks

  • Goodfellow et al, Chapter 6.2-6.5, 4.3

• Lecture 4, 27 Aug 2019: Class notes

  Convolutional neural networks: meaning of convolution; pooling; sharing of parameters; padding;

  • Goodfellow et al, Chapter 9, 9.1-9.3, 9.5

• Lecture 5, 29 Aug 2019: Class notes

  Recurrent neural networks: processing sequential data, unfolding computational graphs, universality of RNNs, teacher forcing for output feedback, backpropagation through time, probabilistic graphical models, RNNs as directed graphical models

  • Goodfellow et al, Chapter 10, upto 10.2.3
  • Russell and Norvig, Chapter 14.1

• Lecture 6, 3 Sep 2019: Class notes

  Recurrent neural networks: Providing context in RNNs; bidirectional RNNs; encoder-decoder architectures, long term dependencies – skip connections and leaky units; LSTM

  • Goodfellow et al, Chapter 10.2.4, 10.3, 10.4, 10.7, 10.9, 10.10 upto 10.10.1
  • Supervised Sequence Labelling with Recurrent Neural Networks, Alex Graves
  • A Critical Review of Recurrent Neural Networks for Sequence Learning, Zachary C. Lipton, John Berkowitz, Charles Elkan

• Lecture 7, 5 Sep 2019: Madhavan's notes (use at your own risk)

  Natural exponential families, Logistic regression, Poisson regression, Generalized Linear Models, Gaussian Process Regression

  Reference: Gaussian Processes for Machine Learning, Carl Edward Rasmussen and Christopher K. I. Williams, Chatper 2: Regression

• Lecture 8, 10 Sep 2019: Madhavan's notes (use at your own risk)

  CNNs and RNNs in Tensorflow: Code for examples shown in class

  Bayesian Optimization, Gaussian Process Regression

  GPR using scikit-learn: Code for Gaussian Process Regression

• Lecture 9, 17 Sep 2019: Madhavan's notes (use at your own risk)

  GPR: Rasmussens' formulation (from Chapter 2 of the RW book)

  Gaussian Process Regression example using scikit-learn, Google Colab notebook

  GPR with large data sets through sampling

• Lecture 10, 19 Sep 2019:

  Bayesian Optimization

  Reference: A Tutorial on Bayesian Optimization by Peter I. Frazier

• Lecture 11, 01 Oct 2019: Madhavan's notes (use at your own risk)

  Bayesian Optimization

  Reference: A Tutorial on Bayesian Optimization by Peter I. Frazier

• Lecture 12, 03 Oct 2019: Madhavan's notes (use at your own risk)

  Autoencoders

  • Goodfellow et al, Chapter 14.1-14.4, 14.6

• Lecture 13, 10 Oct 2019:

  Coding Bayesian optimization

• Lecture 14, 15 Oct 2019: Class notes

  Reinforcement learning: Multi-armed bandits

  • Sutton and Barto, Chapter 2.1-2.7

• Lecture 15, 17 Oct 2019: Class notes

  Reinforcement learning: Markov Decision Processes

  • Sutton and Barto, Chapter 3.1-3.6

• Lecture 16, 22 Oct 2019: Class notes

  Reinforcement learning: Dynamic programming — policy evaluation, policy iteration, value iteration

  • Sutton and Barto, Chapter 4.1-4.5

• Lecture 17, 24 Oct 2019: Class notes

  Reinforcement learning: Monte Carlo Methods

  • Sutton and Barto, Chapter 5.1-5.4

• Lecture 18, 29 Oct 2019: Class notes

  Reinforcement learning: Monte Carlo Methods – off-policy prediction; Temporal-Difference Learning – TD Prediction

  • Sutton and Barto, Chapter 5.5-5.7, 6.1

• Lecture 19, 5 Nov 2019: Class notes

  Reinforcement learning: Temporal-Difference Learning – TD(0), SARSA, Q Learning, Expected Sarsa, Double Learning

  • Sutton and Barto, Chapter 6.1-6.7

• Lecture 20, 7 Nov 2019: Class notes

  Reinforcement learning: Temporal-Difference Learning – n-step Bootstrapping: TD(n), n-step SARSA, n-step off-policy learning, n-step tree backup algorithm

  • Sutton and Barto, Chapter 7.1-7.3, 7.5