Semester page for STK4090 - Spring 2020

The Exam Project is now out on the course website, as of Fri June 12. The deadline is Tue June 23, at 12:12. Your report, qua pdf, then needs to be delivered, i.e. uploaded, to the Inspera system -- where I trust this should work:

https://uio.inspera.no

These course messages have not been kept fully up to date, over the past couple of zooming weeks, since we've been busy with the active and interactive "hjelpetr?der" A, B, C, D, E, F, for and with all course participants.

But check the latest update of the Nils Collection, now with about 100 exercises and 88 pages, and other extra exercises given out in the hjelpetr?der.

I'm also uploading a few com R scripts, related to aspects of some of the Exercises 80-99.

This week, Mon 11.5 and Wed 13.5, we wade through a thousand briges, with the empirical process \rootn (F_n - F) converging to W^0(F(t)), etc., and with consequences for Kolmogorov-Smirnov and Cramer-von Mises and yet other tests.

We're soon done with the entire curriculum, and will utilise the weeks ahead for "consolidation", repetisjon, review of older exercises, testing our muscles, etc. Again, the Exam Project dates are from t_0 = June 12 to t_1 = June 23.

We continue our zoom teaching, with rounds on Mon 20.4 and Wed 22.4, both at 13:00, and for not too long. Check and use also the Hjelpetr?der A, B, C, ...

The Cunen-Hjort FocuStat blog post "Confidence Distributions for Dummies" is curriculum, so read it, and use Hjelpetr?d to ask questions. 

I've uploaded the dataset smallchildren_data to the website. Read in these data in your computer, giving (x1, x2, x3, x4, y), where y is 0-1 for notsmall-small child; x1 is age; x2 is weight before pregnancy; x3 is 0-1 for nonsmoker-smoker; x4 is 0-1 for white-nonwhite.

Carry out logistic regression -- programme the loglikelihood, maximise it, find the standard deviations, find the significant and nonsignificant covariates. Find estimates and confidence intervals for gamma = Pr(Mrs Jones will have a small child), for two or three versions of Mrs Jones (i.e. invent covariates for her).

Dennis and Nils play n = 10^5 iid games, mean zero, finite variance. Let X...

There'll be zoom teaching Wed 15.4 and Mon 20.4 from 13:00, but not very long sessions. Check also Hjelpetr?d 3, and use it actively. Main themes now are

-- consolidation of likelihood things;

-- confidence curves;

-- basics of empirical processes.

-- Extra 1: let x be binomial (n,\theta), with n = 50 and x = 18. First do simple classical things for estimating \theta, with a 90 % c.i. Then make *two* confidence curves for \theta, and draw them in the same diagram: the simple normal approximation; and the one using the deviance function, ccval = pchisq(devval, 1), where you compute the deviance D(\theta) first. Check with Exercise 86.

-- Extra 2: suppose y_1, ..., y_n come from the Cauchy with unknown centre parameter \theta, i.e. with density

f = (1/\pi) / ( 1 + (y-\theta)^2 ). 

Note that the mean is infinite, so the \bar y is hopeless. But there is a ML. Find...

1. As communicated to you on a couple of previous occasions, see also the Hjelpetr?d A, Hjelpetr?d B, ... sequence, we've been allowed by the administration to have our exam project as planned before the korona situation: June 12 to June 23.

2. If anyone is following this course this semester, and might *not* be among the recepients of the Hjelpetr?d mail threads, send me a mail. The same goes for the zoom lectures, where we soon come back to defining a weekly schedule for the time after Easter. Tentatively I also then wish to have *two weekly shorter things*, as opposed to *one three-hour thing each Monday*.

 3. Case in point: we're having a one-hour zoom Monday April 6, from 13:00:

(a) Do Ancient Egypt, Exercise 85, with Weibull instead of Gamma, and the median instead of the mean. Arrive at ML inference, including confidence curves. To what extent can it be inferred that men had longer lives than women? 

(b) Do Exercise 84, also with...

So this particular korona week we do two separate shorter zoom rounds, Mon 30.3 from 13:00 and Wed 1.4 from 13:00, rather than one long three-hour thing.

Check Hjelpetr?der A and B, with more information and details, and, importantly, with good opportunities for yourself to be active, ask questions, etc.

The coming exercises to focus on: from Nils Collection: 85 ("ML in practice!"), the rest of Nils 90, 84, 79.

Check out Version F, of 27 March 2020, of the Nils Collection of Exercises & Lecture Notes. Read through all new exercises (and then attempt to work through them), with themes including likelihood analysis, Kullback-Leibler, profiling, the Wilks theorems, confidence distributions, enpirical processes, Donsker, Kolmogorov-Smirnov.

Enjoy (in these korona times).

1. No "digital lecture" today, infortunately: (a) there's a department allm?te, via zoom, that I need to follow; (b) my 2012 laptop can't properly install zoom or skype; (c) more preparation is needed in order for a new Ipad to work properly for these things.

2. I've posted a mail thread, the Hjelpetr?d A, to all of you or nearly all of you. If you take this course, and for some reason are not among those in the address field for Hjelpetr?d A, send me a mail.

3. Use the Hjelpetr?d A (and later Hjelpetr?d B, etc.) actively -- read, learn, do, accomplish, and post queries.

4. In the course of Mon-Tue March 23-24 I'll write up more Nils Notes & Exercises. There are basically *two more crucial themes* for the course, i.e. in addition to those we've already worked with:

(a) *maximum log-likelihood profiling*, which lead to wondrous *confidence curves*, via Wiilks theorems;

(b) *empirical processes*, with Brownian motion as the limit of partial-su...

Kj?re studenter, h?per alt g?r tilstrekkelig vel med dere og deres, i disse krevende tider. Itj f?rr? n?lles.

1. I'm attempting to make my own half-primitive machinery work for *zoom*, so that we might have a one-hour zoom-lecture Mon 23rd of March. More on this later, but at any rate, check that you do have zoom on your computers.

2. It's been decided that the planned four-hour written exams, in Silurveien etc., will (with high probability) NOT take place in June. But OUR "home project" exam will proceed as planned, in June, and (with high probability) this will be the ONLY exam in this course, with time window [t_0, t_1] = [Fri June 12, Tue June 23]. Again, by definition, more on this later. Other courses will have a "best?tt / ikke best?tt", only, but OUR course will (with high probability) retain the usual scale of A, B, C, D, E, F (which means a premimum for good efforts).

3. By Monday 23.3 I will say something about the latest dozen of N...

1. Itj f?rr? n?lles, to each of you. Look through the 10-15 last exercises in the Nils Collection, and attempt to do some of them, in particular those related to likelihood matters. A separate message will come soon with more on this.

2. It's possible that the teaching Mon Mar 23 and also for a few more weeks will be a mixture of *Zoom* (look it up, install it, check out what it is), several Nils Collection details, and yet other details to come. I'm currently struggling to install and learn basics for Zoom. There'll be a "meeting", digital, with everyone on board tuning in at the same time, with possibilites for "chat" and questions and so on.

3. If you're on Facebook, join the (open) FocuStat group, which has many themes, also related to korona, and to practicalities of teaching, etc.

4. Here's some generic text from the department, today.

Vi pr?ver ? ikke mase for mye, men fant u...

I've uploaded an updated Version E of the Nils Collection of Exercises and Lectures Notes -- print out (when you have a chance to, which in my own case means after the Uni has opened its doors again), enjoy.

Within a few days I will post some thoughts, plans, instructions regarding the efforts we ought to come up with in these El large-sample theory en los tiempos del korona. I will point to some of these new exercises, and also a few from Ferguson's book, for us to go through.

We will also be told within not so many days how "digital teaching" might be arranged over the coming korona weeks.

Corona is upon us, and the UiO has issued various rules of conduct and mechanisms for limiting any spread of the virus. The Department of Mathematics follows suit, and employs *even stricter rules* than a few other departments -- and as a consequence of these there is no teaching for stk 4090 Mon Mar 16.

I lament this (as you do, too), and we'll see what we're able to do over the coming couple of weeks. *At any rate* I'll pretty soon finish up an update of the Nils Collection of Exercises and Lecture Notes, specifically with added material on *likelihood theory*: maximum likelihood, Bayes, profile log-likelihood. Also, I'll give some instructions about what to focus on, and a list of exercises.

I'll look out for certain technical possibilities when it comes to Mon Mar 23. We could I assume also arrange for smaller meetings, with some or all of you (since n \le 8, I think), but possibly avoid calling it "lectures".

Again, in addition to being "gen...

Each course taught with the Mathematics Department needs *a student representative*, to facilitate communication, when necessary, between students & lecturer. Our student representative is *Catharina Stoltenberg*, mail address catharina.stoltenberg kr?llalfa sobaka gmail.com.

It is however & by all means fully possible to communicate directly with Nils Lid Hjort -- talk to him or send him a mail, with any concern for how the course is being taught (etc.).

1. On Mon Mar 2 we rounded off Part III from Ferguson, with a bit more material on sample quantiles, and we started on Part IV, with introductory discussions of maximum likelihood etc. We also did *some* of the listed exercises from Ferguson, and will do more next week. 

2. The course curriculum will be detailed separately, and soon, but consists of five parts -- Parts I, II, III, IV roughly corresponding to these parts of Ferguson's book, and Part V to be on *empirical processes*, with separate material to come from Nils. 

3. For Mon Mar 9, start with the following exercise. First, write out clearly how to form a 95 percent confidence interval for the population median, based on a sample of n datapoints. This involves estimating the limit distribution standard deviation 0.50/f(\mu), and for this ingredient use the kernel density estimate, \hat f(x) = n^{-1} \sum h^{-1} dnorm(h^{-1}(x_i - x)), with default bandwidth taken to be h = 1.059 \sd(x) / n^...

On Mon Feb 24 we went through various exercises, including Scheffe's lemma, and Emil Stoltenberg Exam 2017 no 1, with variance stabilising transformations etc. I then went through Section 13, on sample quantiles and their joint multinormal limit distribution, and this is the single section inside Ferguson's Part III that is defined as core curriculum.  I will also write out some more details qua Nils Exercises in the Nils Collection.

Our forum thanks Ingrid D?hlen for en betimelig og inspirert vits, and Dennis Christensen for an impromptu two-minute lecture to us, on how Euler used the \log(x + \sqrt{1+x^2}) function and its derivative to put up the eternally magical equation

e^{i \pi} + 1  = 0.

https://www.facebook.com/groups/1589206911336271/permalink/2544622505794702/

Next week we start with Part IV, with likelihood inference topics.

For Mon Mar 2, work through Ferguson Exercises 1, 2, 5 from Section 1, 1 from Section 2, 4 from Sect...

1. On Mon Feb 17 I went through a few of the sections of Ferguson's Part II, including the Pearson test statistic (and the limiting chi-squared, 1900) and the correlation story. We also went through Nils Exercises 65, 66, and Emil S exam #3.

2. On Mon Feb 24 I will round of material from Part II, and also do the single hard-core curriculum part from Part III, namely the empirical quantiles. After that we do themes from Part IV, including analysis for ML and Bayes estimators. I'll also upoad the Hjort and Pollard paper to the course site, where the main method, involving convexity, will be defined as inside curriculum.

3. Exercises for Mon Feb 24: Emil Stoltenberg Exam 2016 #1, then Nils Exercises 17, 44, 54.