Week 07: Feb 22-29

Classification and Logistic Regression

Interactive session February 29

• Recording
• Slides
• Menti quiz

Weekly lecture:

Slides

Video Recordings:

1. Linear Regression and Classification
2. The Logistic Function and its Derivative
3. The Logistic Regression Classifier
4. Cross-Entropy Loss
5. Training the Logistic Regression Classifier
6. Variants of Gradient Descent
7. Multi-class Classification: one vs. rest
8. Multinomial Logistic Regression

Readings:

Paolo Perrotta, Programming Machine Learning (in O'Reilly library)

• Chapter 5 "A discerning Machine"
• Chapter 7 "The Final Challenge

Marsland does not cover all the stuff we have considered this week, in particular not Logistic Regression, Multinomial Logistic Regression and loss functions. Perrotta's book, ch. 5 has a useful discussion of logistic regression and loss functions, while ch. 7 presents one-vs.-rest multi-class classification. Ch. 6 "Getting Real" presents the MNIST dataset to be used in ch. 7.

Observe, however, that Marsland, Sec. 4.6, "Deriving Back-Propagation", presents some of the same material as the lecture in the context of multi-layer neural networks (e.g., sec. 4.6.2, 4.6.3, 4.6.5, 4.6.6).

Weekly exercises

• Exercises
• Solution proposals