Wednesday January 25th: After a few words about the course, I started lecturing from Chapter 4 introducing (discrete time, time homogeneous) Markov chains. To illustrate the basic concepts, we looked at examples 4.1, 4.3, 4.4, 4.5, and 4.6. After the break I proved the Chapman-Kolmogorov equations and did examples 4.9 and 4.12.
Friday, January 27th: Covered the material on pages 200-203 about Markov chains killed when entering a subset of the state space. As I got distracted by the smartboard acting up, it's a pretty lousy lecture and in addition the sound is rather bad.
Wednesday February 1st: As I couldn't log into the computer (not even with professional help), I had to use the blackboard. I started by picking up the last remark of Section 4.2 about what happens when the Markov chain doesn't start in a single state, but in a distribution of states, and then proceeded to Section 4.3. I defined communication classes of states and then moved on to recurrence and transience, proving Proposition 4.1 and Corollary 4.2.
Friday February 3rd: I covered the rest of Section 4.3 with an extra emphasis on Example 4.19 on one-dimensional random walks. Unfortunately, the podcast is still (almost) without sound.
Wednesday February 8th: Covered Section 4.4 up to and including (a concrete version of) Example 4.22. In case (iii) on top of page 2.18, I went back to Example 3.16 and sketched the main argument.
Friday February 10th: I first tried to sum up the relationship between expected first return \(m_i\), expected long time occupancy \(\pi_i\), and invariant states \(\pi=(\pi_1,\pi_2,\ldots )\), illustrating it with a simple example. The rest of the time was spent on the probabilistic part of example 4.25, finding the transition matrix and the invariant state. Unfortunately, the sound of the recordings from Auditorium 1 is still bad (I'll send a new complaint). I also seemed to have forgotten to upload the notes (this can be fixed on Monday, I hope).
Wednesday February 15th: I covered subsection 4.4.1 (on a periodic Markov chains), section 4.5.1 (on the Gambler's Ruin problem, including the application to clinical testing on pages 235-237), and section 4.6 on time spent in transient states. Unfortunately, I cannot find the notes from the lecture, but will keep looking for them.
Friday February 17th: Covered section 4.7 on branching processes. Followed the book rather closely, but skipped the derivation of the variance of \(X_n\) as we did not need it. The audio sound is finally ok!
Wednesday February 22nd: Went through section 4.8 on time reversible Markov chains. Covered all the theory, but skipped some of the longer examples. Continues with section 4.9 next time.
Friday February 24th: Covered section 4.9 on Markov Chains and Monte Carlo simulations (only sketched the stuff on Gibbs sampling). There is a little delay in the beginning of the podcast as I had forgotten to take down the screen in the auditorium, and there may be a few short breaks later in the podcast as the recording was interrupted a couple of times. On Wednesday, we start Section 5.2.
Wednesday March 1st: Covered most of section 5.2, including memoryless random variables, propositions 5.1 and 5.2 (with simple examples). We also computed moment generating functions and expectations and variances for the exponential and gamma distributions.
Friday March 3rd: Covered the remaining parts of section 2.2, especially failure functions and sums of exponential random variables (subsection 2.2.4). I dropped subsection 2.2.5 on Dirichlet distributions.
Wednesday March 8th: Started section 5.3 on Poisson process and covered the theory in subsection 5.3.1 and 5.3.2 plus Proposition 5.5 in section 5.3.3. Haven't done any examples yet, but plan to do some on Friday. I had some problems with the in the beginning of the lecture, so please excuse the first few minutes.
Friday, March 10th: I started by reminding people how one can use the CDF to compute expectations of positive continuous and discrete random variables, and then turned to Example 5.17 (I skipped the last part about how many types appear only once). I then took a look at the probability of the n-th event of one Poisson process happening before the m-th event of an another, independent Poisson process (look at the bottom part of page 325/top part of page 326).
Wednesday, March 15th: Covered subsection 5.3.4 up to and including Theorem 5.2 (without proof) and subsection 5.4.1. Unfortunately, the audio is bad again.
Friday, March 17th: Talked about compound Poisson processes (subsection 6.4.2). Most of the time was spent on finding the distribution and the variance. Remember that there are no lectures next week due to midterm exams in other courses.
Wednesday, March 29th: Covered sections 6.1, 6.2, and 6.3. Will start 6.4 next time.
Friday, March 31st: Lectured on section 6.4 up to and including Lemma 6.3, Covered Proposition 6.1, but not the examples.
Wednesday, April 12th: Finished section 6.4 by proving Kolmogorov's forward and backward equations and illustrating them with Proposition 6.4 and Example 6.11 (where I used the matrix methods from MAT1110 to solve the system of differential equations).
Friday, April 14th: Started section 6.5. I followed the derivation of the fundamental equations (6.18) and (6.19) in the book, but then I spent some time working out how these equations look when the state space is finite (equations (6 .18) turns into a simple matrix equation). I even worked out a simple example with three states.
Notes. Podcast (unfortunately without sound)
Wednesday, April 19th: Finished section 6.5 by finding the limit probabilities (when they exist) for birth and death processes. Then I went through section 6.8 on uniformization with example 6.23 as the main application.
Friday, April 21st: Covered sections 6.9 and 7.1. We continue with 7,2 next time.
Wednesday, April 26th: Did section 7.2, plus the random walk part of 10.1 (which I did in more detail than the book). Next time we shall continue with the formal definition (10.1) of a Brownian motion.
Friday, April 28th: Finished section 10.1 by defining Brownian motion and doing a couple of calculations. The first to find the distribution of B(s) + B(t), the second to find the distribution of a Brownian bridge (the calculation at the very bottom of page 641 and top of page 642).
Wednesday, May 5th: Finished the syllabus by going through sections 10.2 and 10.3. Added a little bit about discrete approximations to geometric Brownian motions that is not in the book. Next time I'll start reviewing Chapter 4.
Friday, May 5th: I started reviewing today and covered the topics: Transition matrices, Chapman-Kolmogorov equations, (communication) classes, transience and recurrence, positive recurrence and null recurrence. I didn't quite make it to limit probabilities, but will start there next time.
Wednesday, May 10th: Continued the review by covering sections 4.4, 4.6, and 4.8. Did Problem 1 from Exam 2022 to illustrate most of the points from Chapter 4.
Notes. Podcast. There was a mistake in the formula for \(f_{ij}\) which has now been corrected by a comment in the notes (but not in the recording). As I forgot to stop the recording in the break, there are 15 minutes of absolutely nothing in the middle.
Friday, May12th: For once I did exactly what I had promised: First I did Exam 2017, Problem 2 , then I continued with a brief review of branching processes, and finally I did (most of) Problem 3 from Exam 2020. There is no lecture on Wednesday, May 17th. There will be one on May 19th, but I haven't decided yet whether it will be physical or just a recording.
Friday, May 19th: The lecture has two parts, first a review of exponential distributions and Poisson processes, and then a solution of an exam problem from last year: Problem 2, Exam 2022. (In the review, I for some reason call the parameter of a Poisson process for \(\alpha\) instead of the more conventional \(\lambda\).)
Wednesday, May 24th: Before the break I reviewed the material from chapters 6 and 7, and after the break I did problem 3 from Exam 2021. I had some problems with the smartboard before the break (it was impossible to write straight on the top of the board), and as I didn't dare try to make changes without saving what I had, the notes are in two parts.
Notes, part 1. Notes, part 2. Podcast.
Friday, May 26th. Last lecture. I covered the material on Brownian motion from sections 10.1-10.3 and did problem 3 from the exam of 2018. At the end I said a few words about the exam.