Wednesday, January 25th: The syllabus starts with Chapter 4, and the main purpose of the first lecture is to introduce the concept of a (discrete time, time homogeneous) Markov chain and to see some important and typical examples. Among the examples we shall cover are 4.1, 4.5, 4.6, 4.3, and 4.4 (probably in this order). We shall also take a look at the Chapman-Kolmogorov equations (fancy name for a simple observation) and do examples 4.9 and (if time) 4.12.
Friday, January 27th: We shall continue with the material from section 4.2, trying to pick up the theory that is between all the examples (see the text that starts on the bottom half of page 200, bottom of page 202, and bottom half of page 203), and illustrating it with some examples.
Wednesday, February 1st: I'll start by picking up the last remark of Section 4.2 about what happens when the Markov chain doesn't start in a single state, but in a distribution of states, and then we'll proceed to Section 4.3. This section has a relative simple and intuitive part about classification of states and a more theoretical (and fun!) part about transience and recurrence (that is about whether a Markov chain will visit the same state infinitely many times). We won't be able to cover all of 4.3 on Wednesday, but I hope to cover the rest on Friday. I shall definitely talk about Example 4.19, but not cover the long Example 4.21
Wednesday, February 8th: The topic this week is Section 4.4. On Wednesday I hope to cover the material up to Theorem 4.1 plus a few examples. Most of Friday will be spent on Example 4.25 which contains an important application to genetics. Don't spend too much time trying to get to the bottom of the "proofs" in this section – most of them are not really proofs, but heuristic arguments showing that the results are likely to be true.
Friday, February 10th: I'll start by summing up our results so far and then give a simple example with four states, before turning to Example 4.25 where I will concentrate on finding the transition matrix and the invariant distribution.
Wednesday, February 15th: We shall start by taking a look at subsection 4.4.1 on page 232. This subsection is about periodic Markov chains and will tell us when the distribution of an irreducible, positive recurrent Markov chain converges to the stationary distribution. We then continue with subsection 4.5.1 on The Gambler's Ruin. This is the only part of section 4.5 we'll cover, and once it's finished, we'll continue with section 4.6.
Friday, February 17th: I'll try cover as much as possible of section 4.7 on branching processes (we're slightly ahead of schedule, but I am not going to let that stop me!)
Wednesday, February 22nd: We shall work on Section 4.8 on reversible Markov chains. There are so many examples in the book that we can't look at them all, and after the first couple of pages, I'll concentrate on Example 4.38, Theorem 4.2, and Proposition 4.9. I'll also try to illustrate these results with some shorter examples.
Friday, February 24th: The theme of the day is Section 4.9 on how to use Markov Chains to perform Monte Carlo simulations of random vectors. This is the last section of Chapter 4 that is on the syllabus, and once it is finished, we shall turn to section 5.2.
Wednesday, March 1st: We'll cover as much as possible of section 5.2 on the exponential distribution.
Friday, March 3rd: We'll continue with section 5.2, but what is left is a bit of a mess. I'll pick up the stuff on failure rate functions on page 300. I also hope to have to look at subsections 5.2.4 and (briefly) 5.2.5.
Wednesday, March 8th: We'll start section 5.3 on Poisson processes. A Poisson process counts the number of random events (e.g. the number of accidents) as it develops in time. The connection to exponential distributions is that the time between two events is exponentially distributed. I hope to cover subsections 5.3.1 and 5.3.2 and at least get started on 5.3.3.
Friday, March 10th: As an illustration of the theory of Poisson processes that we have developed so far, I'll do most of Example 5.17 (I'll skip the last part about how many types appear only once). I'll also take a look at the challenge of computing when the n-th event of one Poisson process happens before the m-th event of an another, independent Poisson process (look at the bottom part of page 325), but I may not have time to work out the general formula.
Wednesday, March 15th: I shall take lightly on the rest of section 5.3, skipping subsection 5.3.5 and just talking about (and not proving) Theorem 5.2 from subsection 5.3.4. We shall then continue with section 5.4 on inhomogeneous Poisson process.
Friday, March 17th: The aim is to cover as much as possible of subsection 5.4.2 on compound Poisson processes (I'm not quite sure we will have enough time to cover everything). Note that this is the last part of Chapter 5 that is on the syllabus, and that we shall next turn to Chapter 6.
Wednesday, March 29th: We shall start Chapter 6 on continuous time Markov chains. Section 6.1 is just a brief introduction and Section 6.2 is just a general set-up. The real meat is Section 6.3 on life-and-death processes. As this section uses exponential distributions quite heavily, it may be a good idea to review their basic properties beforehand. Of the longer examples, I plan to do 6.4 and the expectation part of 6.7.
Friday, March 31st: We start section 6.4 from the beginning, covering Proposition 6.1 and (hopefully) lemmas 6.2 and 6.3. The purpose is to prepare the way for Kolmogorov's backward and forward equations.
Wednesday, April 12th: We'll start with proving Kolmogorov's backward and forward equations, and then take a look at some applications, in particular Example 6.11 (which I'll do a little bit differently) and Proposition 6.4 on pure birth processes.
Friday, April 14th: We'll do the theory in Section 6.5 on limiting probabilities. We shall compute the limiting probabilities for birth and death processes, but not look at any of the other examples in the book.
Wednesday, April 19th: We'll first finish section 6.5 by looking at the limit probabilities of birth and death processes and then continue with section 6.8 (sections 6.6 and 6.7 are not part of the syllabus). Here we shall do example 6.23 which is an alternative solution to a problem we have already solved by different methods. I also hope to cover (the quite brief) section 6.9.
Friday, April 21st: We'll do the (quite short) section 6.9 and as must as possible of section 7.1 (also quite short).
Wednesday, April 26th: We shall work on section 7.2. We may have time to start section 10.1.
Friday, April 28th: We'll finish the rest of section 10.1.
Wednesday, May 3rd: The plan is to finish the syllabus by covering sections 10.2 and 10.3 (they are both quite brief).
Friday, May 5th: I'll start the review by trying to make a systematic condensation of sections 4.1-4.5.
Wednesday, May 10th: We shall continue our review of Chapter 4. I also plan to do parts of Exam 2022, Problem 1, and Exam 2017, Problem 2.
Friday, May 12th: I'll first do Exam 2017, Problem 2 which we didn't have time for on Wednesday, and then continue with a brief review of branching processes. Hopefully, I shall have time to do the essential parts of Problem 3 from Exam 2020.
Friday, May 19th: (The physical lecture is cancelled, so this will only be a recording) I'll review chapter 5 on exponential distributions and Poisson processes. At some stage we shall do problem 2 of the Exam from 2022, but I'm not quite sure we shall have time for it in this lecture.
Wednesday, May 24th: I'll review the material from Chapters 6 and 7, and if there is time I'll take a look at Problem 3 from the exam of 2021.
Friday, May 26th: This is the last lecture. I shall quickly review the material from chapter 10 and do problem 3 from Exam 2018. Perhaps I'll also say a few words about the exam.