Plans for week 37

Here's a quick overview of the plans for this week, with a short review of last week.
Last week we started with project 1, which focuses on linear regression (ordinary least squares, Ridge and Lasso regression). The project aims at first fitting a two-dimensional function where we can generate the data. This serves as a stepping stone towards the analysis of real data (maps in three dims).  The reason why we focus on linear regression is that several of these methods have analytical solutions for the optimal parameters. Furthermore, they allow to make links with a statistical interpretation and to discuss central issues in machine learning like resampling methods, overfitting/underfitting (via the bias-variance tradeoff) and more.

Last week during the lectures we discussed how to interpret ordinary least squares and ridge regression from a linear algebra point of view(via the singular value decomposition of a matrix). We ended Friday's lecture with a link to a statistical interpretation. We will continue with the latter this week. We will also discuss  resampling methods like the bootstrap method and cross-validation. With these elements, you should have all that is needed to solve project 1.

The plans for this week are thus
Lab Wednesday and Thursday: work on project 1 
Thursday September 15: Summary of Ridge and Lasso with examples and statistical interpretation. Start resampling techniques
Friday September 16: Resampling methods: Cross-validation and Bootstrap

Recommended Reading:

Lectures on Resampling methods (lectures week 37), see also lectures from week 36, see https://compphysics.github.io/MachineLearning/doc/web/course.html
Bishop 1.3 (cross-validation) and 3.2 (bias-variance tradeoff)
Hastie et al Chapter 7, here we recommend 7.1-7.5 and 7.10 (cross-validation) and 7.11 (bootstrap).
Furthermore, we recommended also the text by Brunton and Kutz. Their videos are of Hollywood quality, and entertaining as well!!
See the excellent videos on the SVD at http://databookuw.com/page-2/page-4/. The texboook by Brunton and Kutz at http://databookuw.com is highly recommended.

Best wishes to you all.
Morten et al.

p.s. if you spot typos, inconsistencies etc in projects, lecture notes, etc, please let us know.
p.s.2 We remind also of the digital labs on Wednesday 1215pm-2pm and 215pm-4pm. The link for these is the same as for the lectures:
Topic: FYS-STK3155/4155 lectures
Time: This is a recurring meeting Meet anytime

Join Zoom Meeting
https://msu.zoom.us/j/91706435521?pwd=Zll6dU1lRVpEbmlMWU9za1dyT0gvQT09

Meeting ID: 917 0643 5521
Passcode: 220382