Plans for week 38, September 15-19

Dear all, welcome back to a new exciting week. We hope you all have enjoyed and are still enjoying the weekend.

The plans this week are to start with a discussion of a statistical interpretation of OLS, Ridge and Lasso (to be continued next week as well). 

This is relevant for the weekly exercises this week and the final part of project 1.  We will also discuss the so-called Bias-Variance tradeoff and resampling methods like cross-validation and bootstrap. The videos listed below may also be helpful. Take a look at them before the lecture if you can. I am particularly fond of the Statquest videos of Josh Starmer, see https://statquest.org/

Material for the lecture on Monday September 15.

1. Statistical interpretation of OLS regression and other statistical properties

2. Resampling techniques, Bootstrap and cross validation and bias-variance tradeoff (this may partly be discussed during the exercise sessions as well.

3. The material we did not cover last week, that is on more advanced methods for updating the learning rate, are covered by its own video. We will briefly discuss these topics at the beginning of the lecture and during the lab sessions. See video on ADAgrad, RMSprop and ADAM (material from last week not covered during lecture) at https://youtu.be/J_41Hld6tTU

Readings and Videos

1. Raschka et al, pages 175-192

2. Hastie et al Chapter 7, here we recommend 7.1-7.5 and 7.10 (cross-validation) and 7.11 (bootstrap). See https://link.springer.com/book/10.1007/978-0-387-84858-7.

3. Video on bias-variance tradeoff

4. Video on Bootstrapping

5. Video on cross validation

For the lab session, the following video on cross validation (from 2024), could be helpful, see https://www.youtube.com/watch?v=T9jjWsmsd1o

Best wishes to you all,

Morten et al