Plans for the week of April 7-11

Dear all, welcome back to FYS5429/9429.  We hope the week has started the best possible way.

We are now in the middle of our discussions on generative models. 

The methods we will focus on are:

1. Energy based models, with the family of Boltzmann distributions as a typical example (last two weeks)

2. Variational autoencoders, based on our discussions on autoencoders (this week and the week after Easter)

3. Generative adversarial networks (GANs) and

4. Diffusion models (after discussions of VAEs)

The last two weeks we have discussed and introduced so-called energy-based models (the recording from last week will be posted soon, I had to remake it since there were parts where the recording went wrong). We will now step into the field of Variational autoencoders (also called Variational Bayesian methods) and develop their math and discuss codes. The VAEs will serve as a stepping stone towards the widely popular diffusion models, before we end our discussion of generative models with GANs.

I hope also to end the course with a series of lectures on the math of reinforcement learning, a very popular method, also based on generating probabilities and Markov Monte Carlo simulations. 

The plan this week is to 

Plans for the week April 8-12, 2024

Generative methods, energy models and Boltzmann machines.

1. Summary of discussions on Restricted Boltzmann machines, reminder from last week

2. Introduction to Variational Autoencoders (VAEs)

Reading recommendations

1. Boltzmann machines: Goodfellow et al chapters 18.1-18.2, 20.1-20-7; To create Boltzmann machine using Keras, see Babcock and Bali chapter 4, see https://github.com/PacktPublishing/Hands-On-Generative-AI-with-Python-and-TensorFlow-2/blob/master/Chapter_4/models/rbm.py

2. More on Boltzmann machines: see also Foster, chapter 7 on energy-based models at https://github.com/davidADSP/Generative_Deep_Learning_2nd_Edition/tree/main/notebooks/07_ebm/01_ebm

3. VAEs: Goodfellow et al, for VAEs see sections 20.10-20.11

We plan also to come with feedback on project 1 before the Easter break kicks in.

In the meantime, all the best and best wishes for the coming Easter break.

The jupyter-notebook for this week is at https://github.com/CompPhysics/AdvancedMachineLearning/blob/main/doc/pub/week12/ipynb/week12.ipynb

Edvin and Morten