WEBVTT Kind: captions; language: en-us NOTE Treffsikkerhet: 91% (H?Y) 00:00:01.200 --> 00:00:08.800 There we go. Okay. So we're going to talk about quality assessment, or in particular critical 00:00:08.800 --> 00:00:13.700 appraisal tools. And for those who are lazy like I am, we very often call them just Cats. It's 00:00:13.700 --> 00:00:21.100 a Cat, but it is a critical appraisal of the methodology of a paper, of a study. That's what we're 00:00:21.100 --> 00:00:27.100 going to talk about. And there is a lot out there. So, let's see how we get this one to the next 00:00:27.100 --> 00:00:28.250 slide. NOTE Treffsikkerhet: 85% (H?Y) 00:00:28.250 --> 00:00:34.100 That's the next slide. So we're going to talk about bias and confounding. We're going to talk about 00:00:34.100 --> 00:00:40.000 study designs at a glance because you will talk about Wednesday little bit more on that. So I will 00:00:40.000 --> 00:00:46.300 just introduce different study designs and then that brings us to critical appraisal tools. I need 00:00:46.300 --> 00:00:53.600 to talk about study designs as well because you start critical appraisal depends on the study 00:00:53.600 --> 00:00:58.500 design. If you talk about it by ICT, randomized control trials, NOTE Treffsikkerhet: 91% (H?Y) 00:00:58.500 --> 00:01:04.750 you've got issues like randomization and that is, of course not the case if you've got an NS1. 00:01:04.750 --> 00:01:09.600 So therefore, you need to know a little bit about research designs before I can talk about quality 00:01:09.600 --> 00:01:15.000 assessment. Still I thought this was important to do this first because this has got in general to 00:01:15.000 --> 00:01:21.050 do what is methodology? What are the pros and the cons? What can we do wrong or good? 00:01:21.050 --> 00:01:28.300 Okay. So let's start with bias and confounding. Now, there is such a thing as I will give NOTE Treffsikkerhet: 74% (MEDIUM) 00:01:28.300 --> 00:01:34.500 you different terminologies different terms and I hope then in the end that works a little bit 00:01:34.500 --> 00:01:42.100 together, but people will talk to you about internal and external validity. I know some of you 00:01:42.100 --> 00:01:48.850 this is all old stuff, but then they can correct me if I do it wrong. Anyhow, internal validity is 00:01:48.850 --> 00:01:55.450 evidence. That is a true and accurate reflection of the participants, procedures and the setting, 00:01:55.450 --> 00:01:58.350 being observed. So just NOTE Treffsikkerhet: 89% (H?Y) 00:01:58.350 --> 00:02:05.800 bit that you are focusing on. External validity has got to do with generalizability. Can I 00:02:05.800 --> 00:02:12.300 generalize the results that I found? Can I generalize the data from my little study to other 00:02:12.300 --> 00:02:17.600 situations? Is the evidence that it is a true and accurate reflection of participants procedures in 00:02:17.600 --> 00:02:24.800 a setting other than the observed? Now, that is usually what you want to do with your research. You 00:02:24.800 --> 00:02:28.250 say, hey, I select 10 schools from Oslo. NOTE Treffsikkerhet: 90% (H?Y) 00:02:28.250 --> 00:02:36.900 And I hope that is representative for the whole of Oslo, maybe even for the whole of Norway. So that 00:02:36.900 --> 00:02:45.500 your data are correct reflection of the actual situation that is internal generalization. That is what we 00:02:45.500 --> 00:02:52.900 always want to do because we can't assess or evaluate all the Norwegians. So you need to select 00:02:52.900 --> 00:02:58.300 wisely and you want that to be representative and that's got to do external validity. NOTE Treffsikkerhet: 82% (H?Y) 00:02:58.300 --> 00:03:06.100 There are always problems in research, that's life. And one of them is bias. 00:03:06.100 --> 00:03:11.800 These are just the definitions, but I'll give you examples later. So, bias is a deviation, 00:03:11.800 --> 00:03:18.800 a difference from the results or interference from the truth. So, something is 00:03:18.800 --> 00:03:25.100 wrong here, or processes leading to this deviation which you call bias and more specifically, it 00:03:25.100 --> 00:03:28.300 can be the extent to which these statistical methods used in NOTE Treffsikkerhet: 91% (H?Y) 00:03:28.300 --> 00:03:36.000 the study do not estimate the quantity thought to be estimated or do not test your hypothesis. 00:03:36.000 --> 00:03:42.000 So it is a problem. That's what you're talking about. Now. I'll give you many examples. You've got 00:03:42.000 --> 00:03:50.700 very different types of publication of bias and one of them is publication bias. So we all know that 00:03:50.700 --> 00:03:58.250 it's much easier to submit a paper that's called ''cure of cancer'' than a paper that says NOTE Treffsikkerhet: 80% (H?Y) 00:03:58.250 --> 00:04:04.300 ''again, we couldn't cure cancer''. The first one will be accepted the second one may just end up in your 00:04:04.300 --> 00:04:10.300 drawer and never be published. So there is absolute publication bias, especially if you've got all 00:04:10.300 --> 00:04:15.800 these small studies. Then the problem is that very often, the small positive studies will be 00:04:15.800 --> 00:04:22.000 published, but the small negative studies just are being hidden. They disappear. So you get a too 00:04:22.000 --> 00:04:26.600 positive image very often. That's publication bias. NOTE Treffsikkerhet: 87% (H?Y) 00:04:26.600 --> 00:04:34.750 Now, the other problem is that and that's got to do with desirability. So respondents very often 00:04:34.750 --> 00:04:39.900 want to, but they do do that deliberately or not, they want they respond, what they think is social 00:04:39.900 --> 00:04:47.900 desirable behavior or social desirable responses, so they want to be accepted. So if you ask, for 00:04:47.900 --> 00:04:56.100 instance, about how much alcohol do you drink? They say, well, not too often, maybe a glass, but we 00:04:56.100 --> 00:04:56.600 know that NOTE Treffsikkerhet: 75% (MEDIUM) 00:04:56.600 --> 00:05:02.000 very, very often these numbers are underrated when you you try to evaluate, when you want to have 00:05:02.000 --> 00:05:09.299 the data. So it's in its regardless of the research format, but people will report inaccuretly. 00:05:09.299 --> 00:05:15.400 I'm speaking too fast. Especially when it's sensitive or personal topics. How many 00:05:15.400 --> 00:05:22.150 partners did you have last year? People will not just blur that out. So that is respondent bias. 00:05:22.150 --> 00:05:26.350 People want to be liked. Now another one is respondent bias, NOTE Treffsikkerhet: 67% (MEDIUM) 00:05:26.350 --> 00:05:32.000 and we talking about for instance sponsored bias, is we all know research is very often 00:05:32.000 --> 00:05:40.500 highly dependent on sponsoring or funding and that means that sponsors, if they get have an 00:05:40.500 --> 00:05:46.900 influence on the output, that's where you get yourself into trouble. So if you've got, if the 00:05:46.900 --> 00:05:51.900 sponsors have got, can actually determine whether for instance, your research being published 00:05:51.900 --> 00:05:56.600 or not... You've got a problem. So sponsoring is good. But also in my area, NOTE Treffsikkerhet: 91% (H?Y) 00:05:56.600 --> 00:06:03.100 many people, I am in solving problems... There are many people that are being sponsored by 00:06:03.100 --> 00:06:08.400 nutritional industries and don't underestimate, that's big money. That's Nutritia, that's Nestle, 00:06:08.400 --> 00:06:15.800 those are very big names. What you don't want to as a research is 00:06:15.800 --> 00:06:21.100 that your name is linked to one industry, because then that is very likely there is some bias 00:06:21.100 --> 00:06:26.550 going on. It's better to either be sponsored by several, but at least they should never NOTE Treffsikkerhet: 74% (MEDIUM) 00:06:26.550 --> 00:06:32.600 ever should have influence on whether you publish your data or on the data or whatever, and it 00:06:32.600 --> 00:06:38.900 is difficult to be independent, unless it is just a research funding because that should be 00:06:38.900 --> 00:06:45.700 independent. So that there is absolutely sponser bias. It's all got to do with money. We've got 00:06:45.700 --> 00:06:52.100 confirmation bias and that is, we've got actually nine out of ten, we've got kind of an idea of what 00:06:52.100 --> 00:06:56.650 we expect to find, we expect to find covid-19 NOTE Treffsikkerhet: 83% (H?Y) 00:06:56.650 --> 00:07:02.400 is bad for Education. We we expect to find that. So if you've got ideas like that in your 00:07:02.400 --> 00:07:09.500 head and we all are guilty of that because we are humans, then you very likely may translate or may 00:07:09.500 --> 00:07:15.500 pick up on those things that fit into your mindset already. So, it seems these beliefs and 00:07:15.500 --> 00:07:22.700 respondents information to confirm that believe. That is if you say the Dutch are 00:07:22.700 --> 00:07:26.500 tall, we are tall, but I walk outside and I will see all these tall NOTE Treffsikkerhet: 87% (H?Y) 00:07:26.500 --> 00:07:35.800 people, whereas there are few. It is your mindset, you look at confirmation. And that is, for 00:07:35.800 --> 00:07:41.700 instance... This is an example, just a simple one, but it is what you expect to see. And these two 00:07:41.700 --> 00:07:48.050 have got different expectations than actually real life, but that's what they see. And my dog is 00:07:48.050 --> 00:07:55.400 absolutely a wolf next door here. Cultural bias, and I can say I'm very different from Norway. 00:07:55.400 --> 00:07:56.450 I think that NOTE Treffsikkerhet: 79% (H?Y) 00:07:56.450 --> 00:08:02.000 We all have got our international SNE course, we've come from different cultures, 00:08:02.000 --> 00:08:10.200 different beliefs, and we have that in our... that's where we grew up, that's what we are used to. 00:08:10.200 --> 00:08:16.800 We compare it with that. We've got this cultural limbs. If I compare the Netherlands to Norway, and I 00:08:16.800 --> 00:08:24.150 always thought we were very, very similar. But now I learned, we talk a lot, we Dutch. And in Norway, 00:08:24.150 --> 00:08:26.400 it's totally different. People are NOTE Treffsikkerhet: 87% (H?Y) 00:08:26.400 --> 00:08:30.900 not just interact in a discussion or not that easy. It's really a different way of 00:08:30.900 --> 00:08:38.200 communicating. So you've got this cultural lens what you expect. Not bad. It's just you need to be 00:08:38.200 --> 00:08:43.500 aware of that it's there. But there are also things that are linked to your research, how you 00:08:43.500 --> 00:08:52.000 design it. And one of them is question order bias. So questions influence each other and that is 00:08:52.000 --> 00:08:56.500 called question order bias. For example, the example here NOTE Treffsikkerhet: 83% (H?Y) 00:08:56.500 --> 00:09:03.500 if respondent rates a product at 10, and then there's a competitive product... You will compare it to 00:09:03.500 --> 00:09:08.400 the 10 you just gave and 10 is the highest, you will see that this is the best. 00:09:08.400 --> 00:09:15.450 You compare it. You always compare to previous order ones afterwards, but also for instance with... 00:09:15.450 --> 00:09:27.400 If you evaluate severity of a NOTE Treffsikkerhet: 82% (H?Y) 00:09:27.600 --> 00:09:34.200 handicap and handicap is wrong word... Disability, that's a better word. If I give you ten videos 00:09:34.200 --> 00:09:41.200 of severely disabled children, and ask how are their hand skills, decode and hand 00:09:41.200 --> 00:09:46.000 coordination, and you rate them. And then I give a child, which is not that severe, you will 00:09:46.000 --> 00:09:50.900 probably rate that one as really positive because you compare with the previous one. So there's 00:09:50.900 --> 00:09:56.500 always this question order bias, not just in the in surveys, but also when you observe NOTE Treffsikkerhet: 81% (H?Y) 00:09:56.500 --> 00:10:05.800 things. We compare with what is previous already rated. Now, this is an example and that's a 00:10:05.800 --> 00:10:14.950 question to you. So if I were to study the prevalence of Cerebral Palsy in school-aged children, 00:10:14.950 --> 00:10:21.100 in Norway. I'm going to do a cross-sectional study, I'm going to do a prevalence study and what I 00:10:21.100 --> 00:10:26.650 will do, I do my subject recruitment in schools for children with special needs in Oslo. NOTE Treffsikkerhet: 89% (H?Y) 00:10:26.650 --> 00:10:31.800 Now, my question to you is, is there bias or not? NOTE Treffsikkerhet: 82% (H?Y) 00:10:31.800 --> 00:10:37.300 And what would be the bias if there is bias, anybody? NOTE Treffsikkerhet: 85% (H?Y) 00:10:44.800 --> 00:10:56.300 Melissa: I don't think there might be a little, because you're only studying this in Oslo. 00:10:56.300 --> 00:11:05.800 So maybe it would be very specific to and you're not really getting a whole range. 00:11:05.800 --> 00:11:16.500 Because Oslo will have different things, that's the first thing stood out for me. 00:11:16.500 --> 00:11:19.200 Renee: I agree with you. There is absolutely biased because if you 00:11:19.200 --> 00:11:24.700 suggest this is Norway, well is Oslo there is representative... So that's one thing we need to 00:11:24.700 --> 00:11:30.600 consider. There's another thing. What do you think about my choice of schools for children with special 00:11:30.600 --> 00:11:36.200 needs? What would be the problem with that? Hana: You're only studying schools with special needs, but you 00:11:36.200 --> 00:11:43.200 want to find the problems in school-aged children. So then it's not a representation of all the children. 00:11:43.400 --> 00:11:49.200 Renee: Exactly. If I go to schools for children special needs, very likely I expect higher 00:11:49.200 --> 00:11:57.200 numbers of prevalence of CP, cerebral palsy. So this is an example of bias. It's not a bad 00:11:57.200 --> 00:12:02.800 study. It's just you're not going to find the answer you were looking for because you chose the 00:12:02.800 --> 00:12:09.900 wrong schools, and you chose a very selected area. Now if your purpose was to determine the 00:12:09.900 --> 00:12:13.500 prevalence of cerebral palsy in schools for children with NOTE Treffsikkerhet: 90% (H?Y) 00:12:13.500 --> 00:12:18.700 special needs in Oslo that's fine with me. Then you're doing exactly what you wanted to know, and maybe 00:12:18.700 --> 00:12:23.599 that's a very feasible and good topic. 00:12:23.599 --> 00:12:29.800 But if you are stating you do the whole of Norway, 00:12:29.800 --> 00:12:35.000 then you've got a problem. Yeah, that's clear for everybody? So, that is why 00:12:35.000 --> 00:12:41.100 also when you write your research proposal, I link your research question to your methods. Do you 00:12:41.100 --> 00:12:43.150 get what you are looking for? NOTE Treffsikkerhet: 91% (H?Y) 00:12:43.150 --> 00:12:48.900 Okay, I'll give you another example. So this I would say, well, this is actually bias. Now I've got 00:12:48.900 --> 00:12:55.500 another one. There is such a thing as speyer therapy. I've got nothing to do 00:12:55.500 --> 00:13:01.550 with it, by the way, but there is such a thing as speyer therapy. So I'm going to study the 00:13:01.550 --> 00:13:08.400 positive effect of speyer therapy. And what's going to happen is I observe, I'm the single 00:13:08.400 --> 00:13:13.450 rater or video recorded therapy sessions. There are problems. NOTE Treffsikkerhet: 88% (H?Y) 00:13:13.450 --> 00:13:15.600 What are the problems here? NOTE Treffsikkerhet: 91% (H?Y) 00:13:15.700 --> 00:13:18.300 Anybody? NOTE Treffsikkerhet: 82% (H?Y) 00:13:20.500 --> 00:13:28.900 One of the problems is how I already formulated my purpose. What is actually a problem in the 00:13:28.900 --> 00:13:31.600 formulation of my purpose? NOTE Treffsikkerhet: 84% (H?Y) 00:13:31.600 --> 00:13:39.300 Anybody? Or is it really a difficult question? Melissa: So could you repeat that? Renee: Okay, what is the problem 00:13:39.300 --> 00:13:45.849 with how I formulated my purpose because that is already a problem in itself. NOTE Treffsikkerhet: 83% (H?Y) 00:13:45.849 --> 00:13:52.700 Kine: If you want to determine because you already think or say it's a positive effect, but you 00:13:52.700 --> 00:13:59.300 expect, you've decided sort of and it should be more 00:13:59.300 --> 00:14:06.300 open or curious. Renee: You are right. Is there an effect? So what the problem is indeed I 00:14:06.300 --> 00:14:14.100 added the word positive. Well, that is my bias and it's like ''I know my therapy is 00:14:14.100 --> 00:14:15.150 fantastic, and I'm NOTE Treffsikkerhet: 82% (H?Y) 00:14:15.150 --> 00:14:21.900 going to prove how positive it is. '' No, you should say, ''to determine the effects of speyer therapy. '' 00:14:21.900 --> 00:14:27.700 You don't say to determine the bad effects or the positive. Now, that is actually bias. 00:14:27.700 --> 00:14:33.500 If you want to know the effects, keep it open. There is some other problems. Also, if I say, I want 00:14:33.500 --> 00:14:40.600 to determine the effects of speyer therapy, and that's my therapy. Then it's not okay. If 00:14:40.600 --> 00:14:45.400 there's a single rater and it's actually me and looking at my own therapy... NOTE Treffsikkerhet: 91% (H?Y) 00:14:45.400 --> 00:14:51.500 That is not very objective enough. So that case I would like, I do like idea of video recording 00:14:51.500 --> 00:14:56.800 therapy sessions, but I would like actually other raters to do whatever they are doing. Yeah, so 00:14:56.800 --> 00:15:01.000 there is a lot of bias actually in this nonsense. By the way, there is such a thing as speyer 00:15:01.000 --> 00:15:09.100 therapy, nothing to do with me. But okay, this is bias. So bias comes in very different 00:15:09.100 --> 00:15:15.150 packages and formulations and it's very difficult not to have bias, but we try when we NOTE Treffsikkerhet: 85% (H?Y) 00:15:15.150 --> 00:15:21.600 do a study to avoid as much bias as we can. Like selection bias, be very careful about where you 00:15:21.600 --> 00:15:28.700 recruit. Don't describe from the start. This, what I expect to find... That's a little bit too much 00:15:28.700 --> 00:15:34.400 focus. Keep it open. You also well, maybe you can't find the results. Then there still could be 00:15:34.400 --> 00:15:39.600 different ways why you don't find the effect, could be your measures are not responsive 00:15:39.600 --> 00:15:45.350 to change, could be your population is too small, could be there is no effect... NOTE Treffsikkerhet: 83% (H?Y) 00:15:45.350 --> 00:15:54.850 But still, keep it open. Now, there is also another issue, another phenomena and that is confounding. 00:15:54.850 --> 00:16:01.800 Confounding is different from bias, but it is also a problem. So confounding is where it is, not 00:16:01.800 --> 00:16:06.800 possible and I always do first the definition, then I'll give you the examples. So confounding is when 00:16:06.800 --> 00:16:13.200 it is not possible to disentangle to get them separate, the effects of two or more processes, 00:16:13.200 --> 00:16:15.150 that are two things going on and NOTE Treffsikkerhet: 81% (H?Y) 00:16:15.150 --> 00:16:21.600 I don't know which one is causing what. And in epidemiology for example, a factor that is associated 00:16:21.600 --> 00:16:28.800 with both risk and exposure status. Now, what does it mean? I'm going to give you examples. We've 00:16:28.800 --> 00:16:38.500 got the sun and the sun is making my plant grow and the sun is also making my ice cream melt. So we 00:16:38.500 --> 00:16:45.450 can say there's a link some with my plant and some with my ice cream. We all agree. NOTE Treffsikkerhet: 85% (H?Y) 00:16:45.450 --> 00:16:54.400 This is not true that my plant makes my ice cream melt, the underlying factor causing the confounding 00:16:54.400 --> 00:16:59.900 factor is the sun. That one is missing here. Yeah, so it's a stupid example, but it's just to 00:16:59.900 --> 00:17:05.200 explain what's happening. So yes, that sun has got, of course there's a relation sun with growth 00:17:05.200 --> 00:17:10.500 and with ice cream that's melting. But there is not a causal, not the connection between that plant 00:17:10.500 --> 00:17:15.300 and my ice cream. I'll give you another example of confounding. NOTE Treffsikkerhet: 86% (H?Y) 00:17:16.000 --> 00:17:25.000 Meet the Dutch. By the way, I am that person on the left. So the Dutch, we eat cheese. Absolutely 00:17:25.000 --> 00:17:32.200 true. We ate a lot of cheese, a lot of cheese. I love cheese. And that's not for my family but 00:17:32.200 --> 00:17:38.300 the Dutch are pretty tall. We really are very, very tall. So there is the Dutch, we eat cheese and 00:17:38.300 --> 00:17:45.750 the Dutch we are tall. Unfortunately, cheese is not making us any taller. It's only making us fat. 00:17:46.250 --> 00:17:52.900 So this is a wrong connection correlation. There's no association between cheese and tall Dutch. NOTE Treffsikkerhet: 88% (H?Y) 00:17:52.900 --> 00:17:59.400 That is confounding. Now. I'll give you something that actually really adapted in our research. So we 00:17:59.400 --> 00:18:07.200 have bolus modification, bolus modification is actually nothing else but we adjust food 00:18:07.200 --> 00:18:12.700 consistencies when you've got swallowing problems. For instance cerebral palsy. We adjust food. We 00:18:12.700 --> 00:18:18.000 won't give you a steak. Maybe we'll make it puree. That's bolus modification. 00:18:18.000 --> 00:18:23.300 We change your food. And we were interested in what is the effect of bolus NOTE Treffsikkerhet: 91% (H?Y) 00:18:23.300 --> 00:18:29.400 modification on health related quality of life. Because if you've got swallowing problems, and I'm 00:18:29.400 --> 00:18:35.000 going to tell you that, I will thicken up your wine for the next 30 years, well that has an 00:18:35.000 --> 00:18:41.400 impact on me. So that is a little bit the idea. If you need to -all your life- 00:18:41.400 --> 00:18:49.600 eat puree food or mashed potatoes and thicken up your glass of wine, that is no fun. 00:18:49.600 --> 00:18:53.150 And it is very, very often NOTE Treffsikkerhet: 84% (H?Y) 00:18:53.150 --> 00:18:58.400 described by therapists. It's so easy. Like, oh, we need to thicken up your liquids. So what is the 00:18:58.400 --> 00:19:04.500 impact of that? The problem was... So, we did two reviews. We did reviews on living with dsyphagia 00:19:04.500 --> 00:19:10.400 swallowing problems and bolus modification, what are the effects and we did also 00:19:10.400 --> 00:19:17.800 quality of life in OD, dysphagia in general. So two reviews. Now the problem with the lower bit 00:19:17.800 --> 00:19:23.300 of our question is that there is a modifier, there is a confounder and that is NOTE Treffsikkerhet: 91% (H?Y) 00:19:23.300 --> 00:19:30.600 the underlying disease. So suppose you've got Parkinson and people with severe 00:19:30.600 --> 00:19:36.800 Parkinson will have bonus modification. They need to adjust the food, but people with severe 00:19:36.800 --> 00:19:43.300 Parkinson, also may have poor quality of life because of all other kinds of reasons. So you can't 00:19:43.300 --> 00:19:50.800 just simply say, bolus modification is resulting in a poor quality of life. This is whole disease 00:19:50.800 --> 00:19:53.199 behind it. Now you could NOTE Treffsikkerhet: 84% (H?Y) 00:19:53.199 --> 00:19:59.100 in studies, you could distinguish that, you could you can add underlying disease in your statistical 00:19:59.100 --> 00:20:06.200 model. The problem is that many studies don't, they just measure what kind of food do people eat and 00:20:06.200 --> 00:20:12.100 they ask ''could you fill in a quality-of-life measure? '' That's not good. That is confounding 00:20:12.100 --> 00:20:15.800 actually. This, the underlying disease is the confounder. NOTE Treffsikkerhet: 87% (H?Y) 00:20:15.800 --> 00:20:22.800 Yep. Okay, and that was actually a problem that we encountered in a study in research. So that 00:20:22.800 --> 00:20:28.100 was not actually the best link. Now. There are a few more terms that you need to be familiar with 00:20:28.100 --> 00:20:35.100 and that is reliability and validity. We will talk much more about psychometrics validity and all of 00:20:35.100 --> 00:20:42.300 that. But for now, we just focus on these two terms and the liability is the degree to which an 00:20:42.300 --> 00:20:45.350 assessment tool produce stable and consistent result. NOTE Treffsikkerhet: 85% (H?Y) 00:20:45.350 --> 00:20:50.800 So I use my measure today, I use my measure in the afternoon and tomorrow and I 00:20:50.800 --> 00:20:58.100 want the same results. If the patient's condition hasn't changed. The other one is validity and 00:20:58.100 --> 00:21:02.300 validity has got to do... and you must have heard these terms but maybe not the definitions. But you 00:21:02.300 --> 00:21:08.700 must have heard about it. Validity is to how well in measure, measures, what it is supposed to 00:21:08.700 --> 00:21:14.000 purported to measure. So, are we actually measuring what we think we're measuring. That's in plain 00:21:14.000 --> 00:21:16.000 English, validity. NOTE Treffsikkerhet: 82% (H?Y) 00:21:16.000 --> 00:21:22.600 If we look at this, it's actually visualized. So on top, we've got these are valid on the right 00:21:22.600 --> 00:21:28.800 because we say it's aiming in the center and you are actually kind of measuring what you wanted to 00:21:28.800 --> 00:21:36.850 measure. But the difference is, this is very precise, meaning reliable. This one is not that precise, 00:21:36.850 --> 00:21:43.200 meaning not that reliable. There's a lot of change there. But if I look on the reliable side, then 00:21:43.200 --> 00:21:45.850 again reliable means consistent. NOTE Treffsikkerhet: 91% (H?Y) 00:21:45.850 --> 00:21:53.200 This is of course the ideal world, very reliable, very valid. This one is still very reliable 00:21:53.200 --> 00:21:59.900 because every time it points at the same direction, unfortunately, it's not valid because you 00:21:59.900 --> 00:22:06.800 were aiming for the center. So it's very precise, but it's not targeting w you wanted to target, 00:22:06.800 --> 00:22:11.900 which is also a problem. And if I go to this one on the left corner down, 00:22:11.900 --> 00:22:15.550 that's actually your nightmare, because we've got here NOTE Treffsikkerhet: 83% (H?Y) 00:22:15.550 --> 00:22:21.500 it's not precise and it's not actually measuring what you want to measure. So that one unit 00:22:21.500 --> 00:22:27.000 must thrown away. That's the worst one. But of course, this is the best one, very precise and exactly 00:22:27.000 --> 00:22:32.750 measuring what you want to measure. This is the difference between validity and reliability. NOTE Treffsikkerhet: 74% (MEDIUM) 00:22:32.750 --> 00:22:38.500 Now, let's go back to this internal and external validity. That is after you've got internal 00:22:38.500 --> 00:22:44.300 validity. That was that you are looking only at the data that you collected. And is that a true and 00:22:44.300 --> 00:22:51.400 accurate reflection of what you want to measure and external was about generalizability. Now, there 00:22:51.400 --> 00:22:57.600 are always threats. There are always problems in the world and there are also in case of validity 00:22:57.600 --> 00:23:02.250 there are some problems. First of all, you've got for internal validity. NOTE Treffsikkerhet: 73% (MEDIUM) 00:23:02.250 --> 00:23:09.000 We've got bias and we discuss that like opinions, expectations, personal believes. Then we've got 00:23:09.000 --> 00:23:14.600 quality of measures and we've got two lectures on psychometrics, and that's got to do with 00:23:14.600 --> 00:23:22.500 reliability, validity, responsiveness. It's got to do with, is my measure reliable, valid, sensitive to 00:23:22.500 --> 00:23:28.900 change. If you measure is not reliable you've got a problem. Then we talk about research 00:23:28.900 --> 00:23:32.250 design. Have you chosen the right research design? NOTE Treffsikkerhet: 69% (MEDIUM) 00:23:32.250 --> 00:23:37.700 If you take any stream or NS1, something like that. Well, that probably will not give you an 00:23:37.700 --> 00:23:45.200 answer to a question about does my therapy work in ASD. That's just a case study. It's nothing 00:23:45.200 --> 00:23:52.200 else. Then other thing is that just depending on your research what you're doing, but in case of 00:23:52.200 --> 00:23:57.700 clinics, very often we've got spontaneous recovery, meaning if you don't have a control group, 00:23:57.700 --> 00:24:02.350 you just have one intervention and you follow them, suppose you just have had a stroke. NOTE Treffsikkerhet: 76% (H?Y) 00:24:02.350 --> 00:24:08.300 And you follow patients the first three months and you give them therapy, 00:24:08.300 --> 00:24:14.600 and say ''I'm fantastic. They are improving. '' Yet, but there's also such a thing as spontaneous recovery. Now, you 00:24:14.600 --> 00:24:21.000 could have dealt with that if you had two groups, one getting your therapy and the other maybe 00:24:21.000 --> 00:24:26.700 usual care or something, then you could have compared the two. But just having one group and you're 00:24:26.700 --> 00:24:31.400 having spontaneous recovery is really undermining your whole study. NOTE Treffsikkerhet: 79% (H?Y) 00:24:31.700 --> 00:24:40.100 And of course what you use to see quite often are the waiting list treatments, that is tough to get that 00:24:40.100 --> 00:24:49.500 one through your ethics, because in how many studies, how many cases is it ethical to say: 00:24:49.500 --> 00:24:54.050 ''Hey, I just identified the problem. We put you on the waiting list. '' NOTE Treffsikkerhet: 81% (H?Y) 00:24:54.050 --> 00:25:01.300 That's a tough one for ethics. Then we've got statistical significance versus clinical significance. 00:25:01.300 --> 00:25:06.850 And the difference is statistical significance you ask Thanassi, that's what he's going to do. 00:25:06.850 --> 00:25:13.600 You've got all these sets and you do your research and say ''I found all these little differences '' 00:25:13.600 --> 00:25:18.400 But then the other reason what you need to ask yourself, the other question is, has it got any 00:25:18.400 --> 00:25:24.199 clinical significance? If we know that the Dutch are 0.1 centimeter NOTE Treffsikkerhet: 87% (H?Y) 00:25:24.199 --> 00:25:30.800 taller than the Norwegians, then what's the meaning of that? Does anybody care? So you need to 00:25:30.800 --> 00:25:35.750 think also, if you are measuring something, the fact that significant, statistically significant... 00:25:35.750 --> 00:25:41.200 Don't just use the word significant for important. I'm talking about statistically significant. Does 00:25:41.200 --> 00:25:49.000 it actually mean that it makes a difference in education or in practice? Okay, and then a 00:25:49.000 --> 00:25:54.100 final comment before we go to the next topic is there is a difference between efficacy NOTE Treffsikkerhet: 87% (H?Y) 00:25:54.100 --> 00:26:01.500 and effectiveness. Efficacy, we refer to it when we are testing 00:26:01.500 --> 00:26:08.700 something just a therapy under optimal conditions, lab conditions, you name it. Effectiveness 00:26:08.700 --> 00:26:14.100 is, well, this is what happens in clinics in normal daily life. Yeah. So efficacy, optimal. 00:26:14.100 --> 00:26:21.600 Effectiveness is what we usually find. Are there any questions so far on this 00:26:21.600 --> 00:26:23.200 topic? NOTE Treffsikkerhet: 86% (H?Y) 00:26:25.400 --> 00:26:34.100 Then I just continue a little bit. Okay, I'm going to get on it. 00:26:49.950 --> 00:26:56.100 We're going to talk about study designs. NOTE Treffsikkerhet: 77% (H?Y) 00:26:56.100 --> 00:27:01.500 I'm just going to show them very briefly to you and Wednesday, 00:27:01.500 --> 00:27:07.800 we will talk more in detail about study designs. Now this is just an example. This is the 00:27:07.800 --> 00:27:14.200 level evidence according to the NHMRC or the national health medical research council. 00:27:14.200 --> 00:27:21.000 And this is what they came up with. Now, you can see at the bottom we have got case series. 00:27:21.000 --> 00:27:26.050 Then we've got comparing studies of NOTE Treffsikkerhet: 90% (H?Y) 00:27:26.050 --> 00:27:32.750 wwo types, we've got RCT's. That is pseudo RCT's, it's almost a randomized control, but just not. 00:27:32.750 --> 00:27:39.100 And then systematic reviews of RCT's. And on top of that clinical guidelines. Clinical 00:27:39.100 --> 00:27:45.600 guidelines should be based on the highest level of evidence available. We talk about all these 00:27:45.600 --> 00:27:51.400 designs a bit later. But the higher in the hierarchy, the higher the level of 00:27:51.400 --> 00:27:56.200 evidence when you use that study design. Certain study designed just give you NOTE Treffsikkerhet: 83% (H?Y) 00:27:56.200 --> 00:28:02.950 very weak evidence. And of course, an RCT gets much better evidence of whether a therapy is working 00:28:02.950 --> 00:28:11.300 than if you just have got 10 patients that you followed up over time. Now there is a reason why 00:28:11.300 --> 00:28:17.900 you choose certain study design. So the purpose of your study, your research question may lead to 00:28:17.900 --> 00:28:23.900 the need to select the certain study design. It's not you can say let's pick any, no, please don't. 00:28:23.900 --> 00:28:26.100 So, effectiveness, NOTE Treffsikkerhet: 91% (H?Y) 00:28:26.100 --> 00:28:31.700 the best ones are controlled trials, but that is for instance, for a master thesis huge challenge, 00:28:31.700 --> 00:28:38.200 that is very difficult to do that. If you talk about, for instance prognosis, we very often follow a 00:28:38.200 --> 00:28:45.100 cohort, a huge group of people over a longer period of times, what are the effects of smoking? You 00:28:45.100 --> 00:28:50.500 need to follow people for years. I've been in Australia for years, what 00:28:50.500 --> 00:28:56.000 are the effects of sunlight? Because man, that sun is intense and they've got a lot of skin cancer going on. NOTE Treffsikkerhet: 91% (H?Y) 00:28:56.000 --> 00:29:01.600 If you want to do a prevalent study, that's cross-sectional, that's one moment only. 00:29:01.600 --> 00:29:08.100 I want to know how many students have got covid today. Then I want to measure today, how many 00:29:08.100 --> 00:29:13.700 people are absent and how many have been confirmed with covid. So depending on your research 00:29:13.700 --> 00:29:19.600 question, you need to select your study design. Sometimes you can choose between different way of 00:29:19.600 --> 00:29:25.400 study designs, they will give you a slightly different answer. But also it should be feasible. NOTE Treffsikkerhet: 83% (H?Y) 00:29:25.950 --> 00:29:32.100 Let's not forget that. So the pros and cons depending on study design to select. Now I'm going to 00:29:32.100 --> 00:29:37.100 give you a few examples of this. 00:29:37.100 --> 00:29:43.900 So first, let's I'll give you the overview. This is an overview of classification of study 00:29:43.900 --> 00:29:49.100 designs. There are different ways of looking at study designs, but in general, you can say, we've got 00:29:49.100 --> 00:29:56.050 descriptive designs and we've got analytical designs. If we are on the analytical side, then we've NOTE Treffsikkerhet: 91% (H?Y) 00:29:56.050 --> 00:30:02.700 got observational versus experimental designs. Okay. What is that? I'll give you some examples. 00:30:02.700 --> 00:30:08.450 Descriptive would be the case report or a number of cases and we're talking about case series, 00:30:08.450 --> 00:30:15.900 cross-sectional, these prevalence studies. Those are all descriptives. If I go to the right, then it 00:30:15.900 --> 00:30:21.200 could be cross-sectional but also more. So depends a little bit but more things like observational 00:30:21.200 --> 00:30:26.300 cohort study, you sit and you observe them for a decade. What is happening? NOTE Treffsikkerhet: 91% (H?Y) 00:30:26.300 --> 00:30:31.800 Case control is you've got two groups, one is exposed, the other one is not and you're going to see 00:30:31.800 --> 00:30:38.900 what happens. Experimental on the other hand is, you introduce an intervention or something. You do 00:30:38.900 --> 00:30:46.300 something, you develop a certain intervention and you have got two randomized groups, 00:30:46.300 --> 00:30:52.600 one gets usual care, the other one gets your intervention. So there are different study 00:30:52.600 --> 00:30:55.900 designs below these descriptive observational and experimental groupings. NOTE Treffsikkerhet: 85% (H?Y) 00:30:55.900 --> 00:31:03.200 Now, I will give you a few examples. Well, I think some people will debate with me, 00:31:03.200 --> 00:31:12.200 but the most simple study design is an N is 1. It's a single case. I don't think it is 00:31:12.200 --> 00:31:17.650 a very good design for a master thesis because you must convince me why you are studying 00:31:17.650 --> 00:31:25.700 one person only for half a year. So I think that's a No-No, of course, in the analogy in real life 00:31:26.050 --> 00:31:31.100 people will publish case reports because maybe and that's usually like ''I've got this very interesting case, 00:31:31.100 --> 00:31:39.100 this child who had ASD and a cochlear implant and how did we deal with it and what happened in 00:31:39.100 --> 00:31:44.700 the next five years and how did the parents with... '' So you can, it's okay. It's just 00:31:44.700 --> 00:31:52.150 for a master thesis it's not the right design, I think it is difficult to make 00:31:52.150 --> 00:31:56.050 sense out of that. Now case series is nothing else NOTE Treffsikkerhet: 76% (H?Y) 00:31:56.050 --> 00:32:01.600 but we've got just a handful of people that we are following. And case series are being used a 00:32:01.600 --> 00:32:09.600 lot in medicine because then for instance, in diseases that are rare, this ALS amyotrophic lateral 00:32:09.600 --> 00:32:15.600 sclerosis and you want to describe a certain intervention in a handful of patients. That's when 00:32:15.600 --> 00:32:23.550 people use case series. So it's very often used for a handful of kind of similar patients or cases 00:32:23.550 --> 00:32:26.050 and you describe what's happening in them. NOTE Treffsikkerhet: 90% (H?Y) 00:32:26.050 --> 00:32:32.400 You actually don't use statistics. The group is too small, you simply report on it. I also don't think 00:32:32.400 --> 00:32:39.500 it's the best way for a master thesis. But again, you could try to convince me. Now this is also, I 00:32:39.500 --> 00:32:46.500 gave you some examples that you will not do, this is another one. A cohort is, we've got 00:32:46.500 --> 00:32:54.100 here, two groups. We've got a group of people that are overweight and we've got a group of people 00:32:54.100 --> 00:32:56.000 with normal weight and then we NOTE Treffsikkerhet: 72% (MEDIUM) 00:32:56.000 --> 00:33:01.300 gonna sit for five years or whatever and we just see, well, what are the problems in 00:33:01.300 --> 00:33:08.300 each group and we compare the risk factors. So, for instance, the group with overweight 00:33:08.300 --> 00:33:15.700 may have more cardiovascular problems compared to the people without 00:33:15.700 --> 00:33:23.350 overweight they may have more diabetes. That is a cohort. Cohorts are usually big numbers. I will 00:33:23.350 --> 00:33:26.000 follow the cohort SNE students, NOTE Treffsikkerhet: 87% (H?Y) 00:33:26.000 --> 00:33:30.300 not that big but okay, and I follow them for the next 10 years. And I want to know where they end 00:33:30.300 --> 00:33:36.700 up with. Yep. So that is a cohort. Also, because you need to follow usually big groups in over a 00:33:36.700 --> 00:33:43.750 longer period of time, it doesn't seem to be a right study design for you. Cohort is sometimes 00:33:43.750 --> 00:33:50.200 abused for a very small groups, but I would not call that a cohort. Cohort is everybody born in 00:33:50.200 --> 00:33:55.950 1963, that's a cohort. It's not just the five people living in my street. That's not a cohort. NOTE Treffsikkerhet: 76% (H?Y) 00:33:55.950 --> 00:34:01.250 That's too small. Use it for big groups. So it's a group of people with a shared 00:34:01.250 --> 00:34:07.600 characteristics and you follow them over time, but there are different groups within this cohort. 00:34:07.600 --> 00:34:13.400 There are overweight one and normal weight, but you just follow them. Now this is case control and 00:34:13.400 --> 00:34:17.000 that's working backwards in this case. So what we're doing here, we've got a group of cancer 00:34:17.000 --> 00:34:18.400 patients. NOTE Treffsikkerhet: 83% (H?Y) 00:34:18.400 --> 00:34:24.400 I've got a group of no cancer patients and we look in the histories and compare them and see what 00:34:24.400 --> 00:34:29.500 are the differences. So maybe cancer patient did smoke a lot. Maybe cancer patients got in 00:34:29.500 --> 00:34:35.800 touch with materials that they should not have been in touch with, maybe it's a family component, 00:34:35.800 --> 00:34:40.400 all of that. So you've got two groups and you want to understand what could have caused the 00:34:40.400 --> 00:34:47.300 difference. What could possibly have been risk factors that one group has got the disease or the condition 00:34:48.199 --> 00:34:57.500 and the other group has not. That is a case control study. So first you've got cases, meaning 00:34:57.500 --> 00:35:04.300 participants, with a certain outcome, and controls and those are participants without outcome, and you 00:35:04.300 --> 00:35:08.700 need to be sure that they are furthermore comparable. So you don't want to have your cancer 00:35:08.700 --> 00:35:15.500 patients, like they are 80 plus and your control group is 20 years, that doesn't make sense. They 00:35:15.500 --> 00:35:18.250 need to be comparable. You don't want one NOTE Treffsikkerhet: 91% (H?Y) 00:35:18.250 --> 00:35:24.100 only men and the other only women. So you need to have a look at what are the... also in kids you 00:35:24.100 --> 00:35:29.500 don't from all boys or and the other one all girls. It should be comparable, those groups. And then 00:35:29.500 --> 00:35:35.600 you can look back. Like, okay, what is different there? You go into history. That is for an example 00:35:35.600 --> 00:35:40.850 of a case control, looking for to identify risk factors. NOTE Treffsikkerhet: 81% (H?Y) 00:35:40.850 --> 00:35:46.600 Now, this is beautiful. And that's also not going to happen. That's a randomized control trial. That 00:35:46.600 --> 00:35:54.200 is the top highest level of evidence. We really need more, because what you do, you start and 00:35:54.200 --> 00:36:02.200 they can be anything. It can be children with social behavioral issues with 00:36:02.200 --> 00:36:07.000 problems, but you've got your group with whatever condition you looking at. NOTE Treffsikkerhet: 82% (H?Y) 00:36:07.000 --> 00:36:15.649 And then add random and that's important at random, tese participants are being assigned an 00:36:15.649 --> 00:36:22.700 intervention, and it could be a treatment or it could be a placebo or it could be a 00:36:22.700 --> 00:36:28.100 waiting list or it could be no therapy or usual care about something like that. But this is your 00:36:28.100 --> 00:36:34.600 intervention compared to a group that will not receive your intervention. Usually it's a no 00:36:34.600 --> 00:36:36.700 therapy group or placebos, NOTE Treffsikkerhet: 88% (H?Y) 00:36:36.700 --> 00:36:41.400 that they say, okay, one gets the medication, the other one gets a 00:36:41.400 --> 00:36:50.200 placebo medication. But it can be more more complex if you say, just finished another review on 00:36:50.200 --> 00:36:56.600 again on swallowing. So we have people that actually use neurostimulation. So, putting electrodes 00:36:56.600 --> 00:37:01.200 and putting some simulation on it. And then they give the other groups, that's the intervention, and 00:37:01.200 --> 00:37:06.550 the other groups group get Shem stimulation, meaning something happens, but they NOTE Treffsikkerhet: 77% (H?Y) 00:37:06.550 --> 00:37:14.800 expect that it's not enough to make a difference. So, again, random allocation is essential in a 00:37:14.800 --> 00:37:21.000 randomized controlled trial, then one gets a treatment, the other one not and then you compare the 00:37:21.000 --> 00:37:28.300 outcomes, that is a beautiful RCT. Very often you need a lot of resources. It is very often 00:37:28.300 --> 00:37:36.650 expensive, but it is a very nice way of doing research. So again, randomly assigned NOTE Treffsikkerhet: 86% (H?Y) 00:37:36.650 --> 00:37:42.300 two or even more interventions, you can have more. Each gets a different intervention or no 00:37:42.300 --> 00:37:48.600 intervention. Groups need to be comparable. You really have got a problem if one is again, there are age 00:37:48.600 --> 00:37:54.500 difference or the severity of the underlying problem is very different in both groups... NOTE Treffsikkerhet: 79% (H?Y) 00:37:54.500 --> 00:38:02.100 And then you compare them. And as you can see that can be here no treatment, placebo treatmenti you 00:38:02.100 --> 00:38:06.600 don't tell the patients. They do know there's a chance they get a placebo treatment, you need to 00:38:06.600 --> 00:38:13.800 explain it. But Placebo or Sham and that's anything that's lacking activity, but it looks like 00:38:13.800 --> 00:38:20.500 they're getting an intervention and that is a randomized control trial. Now I think I'm going to 00:38:20.500 --> 00:38:23.500 stop right here. NOTE Treffsikkerhet: 91% (H?Y) 00:38:26.500 --> 00:38:34.800 Here we are. So we talked about the rcts or randomized control trial. Now, randomized control 00:38:34.800 --> 00:38:41.700 trials as we know is all about these randomly assigned groups, two or more, it can have more 00:38:41.700 --> 00:38:48.200 arms, each group is called arms. But each group will have a different either a 00:38:48.200 --> 00:38:53.700 different intervention, could be a different intensity but something is different, and they 00:38:53.700 --> 00:38:57.100 should be comparable groups, but usually the control group NOTE Treffsikkerhet: 86% (H?Y) 00:38:57.100 --> 00:39:02.600 is no treatment or Placebo or Sham, something like that. NOTE Treffsikkerhet: 85% (H?Y) 00:39:02.600 --> 00:39:07.800 But these are more complex designs. And again, I don't expect you to do that for your master 00:39:07.800 --> 00:39:14.500 thesis. For instance if you look at this, we've got a 2 by 2 factorial clinical trial. That can be 00:39:14.500 --> 00:39:20.550 any trial. But you've got a treatment A here on the left, and a treatment B on 00:39:20.550 --> 00:39:27.600 the top, and any combination is in this design. Meaning, I've got a group that receives both 00:39:27.600 --> 00:39:33.050 treatments, a group that receives none of these treatments, one group gets only A NOTE Treffsikkerhet: 88% (H?Y) 00:39:33.050 --> 00:39:42.149 and one group gets only B, and all of those groups together, then you can compare like this is B 00:39:42.149 --> 00:39:47.900 and the other group here horizontal have got all this A. You can have a look at it, 00:39:47.900 --> 00:39:55.100 but for different conditions, having a 2x2 factorial clinical trial. Now that is quite complex. 00:39:55.100 --> 00:40:00.100 You've got also another one. But this is about two or more interventions. This is just two by 00:40:00.100 --> 00:40:02.950 two, but it can be more complex and you want to NOTE Treffsikkerhet: 81% (H?Y) 00:40:02.950 --> 00:40:08.400 have all these combinations in your design and you want to know whether each intervention is 00:40:08.400 --> 00:40:14.500 efficacious or also whether a combination is doing a better job. Sometimes you don't know. Should I 00:40:14.500 --> 00:40:23.600 get A or B or A and B, that's what you can do here. That's what you can check. Now, other thing 00:40:23.600 --> 00:40:30.100 what people sometimes do is cross-over, and that means again, I've got study participants, 00:40:30.100 --> 00:40:33.050 I randomize them for treatment A or treatment B, NOTE Treffsikkerhet: 83% (H?Y) 00:40:33.050 --> 00:40:40.800 whatever that is, then I've got a so-called wash out period and that is a period that you say: 00:40:40.800 --> 00:40:47.000 ''Okay, let's wait until the effects of the treatments are no longer doing their their bits. There are 00:40:47.000 --> 00:40:54.300 no longer influencing. '' And then I swap them. So then this group, A gets now treatment B, and B gets A. 00:40:54.300 --> 00:41:02.100 Now that happens we talk about a cross over study design. So, each group gets both 00:41:02.899 --> 00:41:06.400 treatments but in a different order. NOTE Treffsikkerhet: 91% (H?Y) 00:41:06.400 --> 00:41:13.050 Okay, so they intentionally cross over to the other intervention on. That's what you're going to do. 00:41:13.050 --> 00:41:19.900 So everybody first gets one intervention, usually random allocation that makes it much stronger and 00:41:19.900 --> 00:41:26.900 then you switch after that wash out period. Now these one nothing but examples and we will talk 00:41:26.900 --> 00:41:34.300 again on Wednesday more about this. But why would we even care about study designs? Because 00:41:34.300 --> 00:41:36.000 different types of questions are NOTE Treffsikkerhet: 91% (H?Y) 00:41:36.000 --> 00:41:43.500 best answered by different types of study designs. So if you are preparing for your research 00:41:43.500 --> 00:41:51.200 proposal and for your master thesis, you need to select a study design that is feasible but of 00:41:51.200 --> 00:41:58.700 course also fits your research question, and the criteria,you used to praise your research will 00:41:58.700 --> 00:42:06.000 vary depending on your study design and that is what brings us to critical appraisal tools or our Cats. 00:42:08.150 --> 00:42:16.700 Now, what is the aim of a Cat? What is the aim of critical appraisal? What you do is you want to 00:42:16.700 --> 00:42:24.350 identify threats. You want to know what could possibly -in simple English- mess up my research, 00:42:24.350 --> 00:42:31.300 you're looking for the problems in your design. You look at threats to validity of your research 00:42:31.300 --> 00:42:36.700 findings from the literature and you provide consumers of research evidence the opportunity to make 00:42:36.700 --> 00:42:37.649 informed decisions NOTE Treffsikkerhet: 83% (H?Y) 00:42:37.649 --> 00:42:43.050 about the quality of research evidence for. There's a lot but what you are doing is actually 00:42:43.050 --> 00:42:49.700 you are looking at the article and step-by-step you go through this study and you look at 00:42:49.700 --> 00:42:56.600 is there selection bias, is there is device that's misused, is the methods okay, all of that and you've 00:42:56.600 --> 00:43:00.950 got tools for that and you call those critical appraisal tools. NOTE Treffsikkerhet: 83% (H?Y) 00:43:00.950 --> 00:43:07.000 Now, the critical appraisal is also it helps 00:43:07.000 --> 00:43:13.300 you how to write. So, what you would do, you identify most relevant papers and it is to 00:43:13.300 --> 00:43:19.400 distinguish evidence from opinion and some options, misreporting. It is about assessing the validity 00:43:19.400 --> 00:43:24.600 and assessing the usefulness and clinical applicability of the study, the generalizability of your 00:43:24.600 --> 00:43:29.500 study. How do you do that? You do that with Cats. NOTE Treffsikkerhet: 79% (H?Y) 00:43:29.500 --> 00:43:36.600 Now, I'll give you an example. Validity of a study, for instance if we talk about a treatment for example 00:43:36.600 --> 00:43:43.700 talking about an RCT, then you would like to know, are the patients randomized? You would like to 00:43:43.700 --> 00:43:48.700 know what's the allocation concealed, meaning, that the patient's know they were going to a 00:43:48.700 --> 00:43:54.800 placebo or not. That's quite important. It will influence. Are the baseline 00:43:54.800 --> 00:43:59.600 characteristics the same? So gender distribution, NOTE Treffsikkerhet: 83% (H?Y) 00:43:59.600 --> 00:44:07.100 age distributions, severity of the disorder, is that kind of comparable? If one is much 00:44:07.100 --> 00:44:13.600 more, is more severe than the others how can you can compare in the end? It's blinded. 00:44:13.600 --> 00:44:21.000 What blinded means? For instance, if I get data from your RCT, I do my analysis and I 00:44:21.000 --> 00:44:27.500 don't know what group is receive the intervention. So I only know I've got an A, and a B group, but 00:44:27.500 --> 00:44:29.550 I don't know which one was the actual intervention. NOTE Treffsikkerhet: 91% (H?Y) 00:44:29.550 --> 00:44:34.600 That is blinded for me, but it can also that participants, they should don't 00:44:34.600 --> 00:44:41.000 know which arm they are assigned to. So that has all got to do with blinding. 00:44:41.000 --> 00:44:48.400 You will have a look at follow-up completeness, because what happens very often is that authors 00:44:48.400 --> 00:44:56.200 write in that abstract '' We had 400 participants. '' Yeah, but the statistics was maybe only 100, 00:44:56.200 --> 00:44:59.399 then ignore the 400, your NOTE Treffsikkerhet: 75% (MEDIUM) 00:44:59.399 --> 00:45:06.900 results are based on 100. So you always look at what is the statistic based on, that is the final 00:45:06.900 --> 00:45:13.500 number that you need to post on data set and it's very normal in research that we lose some 00:45:13.500 --> 00:45:18.500 patients. Lost to follow-up, they didn't want to do it anymore, they walk away, 00:45:18.500 --> 00:45:25.800 that is normal. But if you report on ''we did a study in an X number of participants'', I want to 00:45:25.800 --> 00:45:29.500 know how many participants completed the intervention. NOTE Treffsikkerhet: 75% (MEDIUM) 00:45:29.500 --> 00:45:36.000 And that has got to do with validity, Etc, and reliability of your study. So is there an 00:45:36.000 --> 00:45:41.800 intention-to-treat? So that has got to do with follow-up and all of that, but that 00:45:41.800 --> 00:45:48.700 was just in general. Now, if we look at etiology for instance, and we're interested in what 00:45:48.700 --> 00:45:55.800 is the cause of it, and we could do a cohort or case control study. Then again comparison groups, 00:45:55.800 --> 00:45:59.550 always comparison groups needs to be similar at baseline, NOTE Treffsikkerhet: 86% (H?Y) 00:45:59.550 --> 00:46:06.300 the characteristics, the patient characteristics or the study population characteristics. Are the 00:46:06.300 --> 00:46:12.300 outcomes and exposures measure the same for both popular groups? So did they received the same 00:46:12.300 --> 00:46:19.050 intervention, you need to know the dosage of that and again was the follow-up complete? NOTE Treffsikkerhet: 89% (H?Y) 00:46:19.050 --> 00:46:26.600 So every study design has got his kind of some things are overlapping, some things are items are 00:46:26.600 --> 00:46:34.100 particular for that specific type of research. Qualitative research, are the aims of the research 00:46:34.100 --> 00:46:40.300 clearly stated, was the methodology appropriate and usually you get criteria that you can read like 00:46:40.300 --> 00:46:47.000 what is appropriate and what is not appropriate. How did they recruit, was that okay, were the data 00:46:47.000 --> 00:46:49.400 collected in a way that adress the research, NOTE Treffsikkerhet: 89% (H?Y) 00:46:49.400 --> 00:46:54.700 issue like that. They just collect any data or had these strategies for that. What is the 00:46:54.700 --> 00:47:00.700 relationship between research and participants? Is there bias, is there selection bias or anything like that. 00:47:00.700 --> 00:47:09.350 And what about the data analysis used, again you need Cats to make this happen. Cats are 00:47:09.350 --> 00:47:17.700 structured forms for different study designs. And they lead you how to evaluate methodological 00:47:17.700 --> 00:47:19.350 quality of studies. NOTE Treffsikkerhet: 82% (H?Y) 00:47:19.350 --> 00:47:27.300 Yeah, and this is one of them. So this is for instance if we look at a single case study, you've 00:47:27.300 --> 00:47:33.700 got also Cats for single case studies. A single case that is the N is 1 trial, and that is the 00:47:33.700 --> 00:47:40.100 consort extension for reporting N of 1 trials, 2015 statement. They've got all these beautiful 00:47:40.100 --> 00:47:48.900 names, in short set. Now you can see here there is a very often these cats have got websites 00:47:49.400 --> 00:47:56.700 we can see more information about it. So just to give you an example. So this is a single 00:47:56.700 --> 00:48:02.400 case study. We've got N of is 1. It's about an experimental clinical study design to 00:48:02.400 --> 00:48:08.200 determine the effect of an intervention in a single study participant. Remember, we are at the 00:48:08.200 --> 00:48:16.100 bottom of a hierarchy, this low evidence, but you can do that. And then would you use is this Cat 00:48:16.100 --> 00:48:19.600 which will help you a little bit. NOTE Treffsikkerhet: 86% (H?Y) 00:48:19.600 --> 00:48:26.700 So that is about an experimental clinical study. You've got one single case and sent has got for 00:48:26.700 --> 00:48:33.300 each part of that article. It has got items, for instance title and Abstract sense as well. Then 00:48:33.300 --> 00:48:37.550 you should say that in your title. You should mention that we're talking about a case study. 00:48:37.550 --> 00:48:43.400 Introduction, you need to give it a rationale, and that is what I also tell to you students very 00:48:43.400 --> 00:48:49.500 often: Why do you do a N is 1? Why did you not do 5 NOTE Treffsikkerhet: 76% (H?Y) 00:48:49.500 --> 00:48:59.300 Or 10 or 20? And then it's very weak to say I ran out of time, half a year, hmm. So it is 00:48:59.300 --> 00:49:06.400 make sense that you say a rationale of N is 1. And that's why N is 1 very often used 00:49:06.400 --> 00:49:13.600 for special case descriptions. Methods, and then we talked about the trial design, the participants 00:49:13.600 --> 00:49:19.000 recruitment, the interventions. And you got all these criteria that you need described like, NOTE Treffsikkerhet: 82% (H?Y) 00:49:19.300 --> 00:49:24.700 okay, what have they done, did they describe the trial design, the plant number of periods, the 00:49:24.700 --> 00:49:29.800 duration of each period, how often did they measure? Because very often it's repeated measurement. 00:49:29.800 --> 00:49:38.500 What they followed, one case over time... What about the participants, the diagnosis of 00:49:38.500 --> 00:49:45.400 disorder... The criteria, like, if you say ASD. Well, what are criteria? How do you 00:49:45.400 --> 00:49:49.300 diagnose ASD? Autism spectrum disorders. How do you do decide NOTE Treffsikkerhet: 85% (H?Y) 00:49:49.300 --> 00:49:56.100 these participants meet those criteria, who confirmed that diagnosis? Because again, you want 00:49:56.100 --> 00:50:02.300 to make sure that you are actually having the right disciplines there. And this is just for an N is 1. 00:50:02.300 --> 00:50:04.200 So that is said. NOTE Treffsikkerhet: 77% (H?Y) 00:50:04.200 --> 00:50:09.900 You also got other Cats and I just will show you a number you don't need to learn them by heart, 00:50:09.900 --> 00:50:15.600 but that you know you've got these different ones. So Strop is for instance, for cross-sectional 00:50:15.600 --> 00:50:23.500 studies. It's short for strengthening the reporting of 00:50:23.500 --> 00:50:29.200 observational studies in epidemiology. And again, this is website again where you can see it 00:50:29.200 --> 00:50:34.750 and it is one moment of measurement and you want to know for instance, the prevalence of 00:50:36.250 --> 00:50:44.000 shyness in eight year olds. I measure it today. Something like that. Now, 00:50:44.000 --> 00:50:49.600 how does it look? Very often you can see the same structure. You've got here the criteria. 00:50:49.600 --> 00:50:55.000 This is the checklist that you just need to scroll down. And for each thing like study design and 00:50:55.000 --> 00:50:59.700 setting, they say, well, did they describe all of that correctly, did they describe the setting, the 00:50:59.700 --> 00:51:06.250 locations, did they describe how they were going to address potential sources of bias, did they NOTE Treffsikkerhet: 90% (H?Y) 00:51:06.250 --> 00:51:13.000 explain how the study size was arrived at, and you get in this of course there is more information for each 00:51:13.000 --> 00:51:18.900 Cat at the website. This is just in short, then they've got criteria about the study participants, 00:51:18.900 --> 00:51:25.200 again, describe how did they describe the participants, the diagnosis and they've got this for 00:51:25.200 --> 00:51:32.100 different type of study designs, they give you information what they want. So all these criteria you 00:51:32.100 --> 00:51:35.850 just tick off. Same with outcomes. NOTE Treffsikkerhet: 91% (H?Y) 00:51:35.850 --> 00:51:42.900 I see that very, very often people use outcome measures that are not valid or reliable. 00:51:42.900 --> 00:51:49.300 That is a disaster in my mind because then you can't interpret your outcome. If you use a measure, 00:51:49.300 --> 00:51:57.700 that is not valid, why using it? Usually very often, it is unknown. I can live with unknown 00:51:57.700 --> 00:52:03.800 psychometrics. I cannot live with measures that have been proven to be not valid. That's a waste of 00:52:03.800 --> 00:52:06.149 your energy. But anyhow, NOTE Treffsikkerhet: 82% (H?Y) 00:52:06.149 --> 00:52:12.100 you've got the outcomes, eligibility criteria, which means that very often you need to 00:52:12.100 --> 00:52:18.600 explain ''This is where we recruited. These were the criteria. These were the people we excluded. 00:52:18.600 --> 00:52:24.500 These are the people that we lost during follow-up '' , and very often it comes with a flow diagram. 00:52:24.500 --> 00:52:31.100 Just give them the overview of ''We start with the 100 that only 40 enrolled, 10 were 00:52:31.100 --> 00:52:36.100 lost to follow-up. We've got complete data set for 25. Something like that. '' You want to know that. 00:52:37.400 --> 00:52:45.000 Now, this is an example of one that we did. We did cross a cross-sectional study. It was actually a 00:52:45.000 --> 00:52:50.800 review that we did and we wanted to know the prevalence, that's a cross-sectional study, of 00:52:50.800 --> 00:52:58.050 swallowing problems in a number of groups, a number of diseases, Parkinson, 00:52:58.050 --> 00:53:03.900 Alzheimer's, Etc. Because we know there's a lot of strong problems in these groups. So we retrieved 00:53:03.900 --> 00:53:07.350 all the literature but then every single study that NOTE Treffsikkerhet: 78% (H?Y) 00:53:07.350 --> 00:53:15.300 ported on prevalence, we used these criteria called axis. 00:53:15.300 --> 00:53:22.100 So that was cross sectional studies and Axis has got 20 items and you can you mark each 00:53:22.100 --> 00:53:29.400 item as yes, no, do not know. Maximum is 20, minimum is 0. So each article that you include in your 00:53:29.400 --> 00:53:35.300 systematic review, for instance need to have been assessed with a Cat, in this case because we were 00:53:35.300 --> 00:53:37.350 only interested in cross sectional, NOTE Treffsikkerhet: 87% (H?Y) 00:53:37.350 --> 00:53:44.600 we could use the Axis. And this is just an endless list. But again, 00:53:44.600 --> 00:53:52.900 you can see here introduction methods, results and discussion. So very often it follows the 00:53:52.900 --> 00:53:59.800 order, the recipe of how to write a study, how to write it a paper and that makes sense. You can 00:53:59.800 --> 00:54:02.000 see that in most Cats. NOTE Treffsikkerhet: 91% (H?Y) 00:54:02.000 --> 00:54:08.600 Okay, back to the control trials. These are the randomized control trials. There is something that's 00:54:08.600 --> 00:54:17.500 called a consort statement and that is again, short for Consolidated Standards of Reporting Trials, 00:54:17.500 --> 00:54:22.500 reporting of randomized control, trials. I don't know who comes up with these endless names but it 00:54:22.500 --> 00:54:29.250 is consort in short. And we talk about RCT's. So, if you want to know about the methodological 00:54:29.250 --> 00:54:32.100 quality of your RCT study, NOTE Treffsikkerhet: 91% (H?Y) 00:54:32.100 --> 00:54:39.000 this is what you use. And again, you can see --and I'll make it a little bit bigger, but you can see 00:54:39.000 --> 00:54:46.300 already-- title and Abstract, intro, methods, --and there we go, and I make it a bit bigger.-- Same, this the 00:54:46.300 --> 00:54:51.600 same content, it's the title, abstract about background, objectives as they have 00:54:51.600 --> 00:54:57.800 been described, then they talk about design, participant, interventions and outcome measures, user sample, 00:54:57.800 --> 00:55:01.950 sizes on adequate number. Have they been a trend randomized and blinded? NOTE Treffsikkerhet: 77% (H?Y) 00:55:01.950 --> 00:55:08.900 And are the stats correct? And again, for each item in these Cats, you will have always a manual 00:55:08.900 --> 00:55:11.550 that explains what you need to have a look at. NOTE Treffsikkerhet: 85% (H?Y) 00:55:11.550 --> 00:55:17.800 And this is the same, the second bit and you can just move to the results with the talk about. 00:55:17.800 --> 00:55:23.800 This what I previously said, this diagram that very often is included to say ''we start 00:55:23.800 --> 00:55:32.500 with the 100 patients, 10 died, 10 walked away. We had a crash computer, another 10 loss. '' That's 00:55:32.500 --> 00:55:38.600 what you need to, give the flow chart and at the bottom you say these are the patients that we 00:55:38.600 --> 00:55:41.150 have pre and post data for. NOTE Treffsikkerhet: 91% (H?Y) 00:55:41.150 --> 00:55:48.200 Again, recruitment based on data numbers, analysis -- this is a little bit if you've got 00:55:48.200 --> 00:55:55.900 additional analysis. -- I'm not going to get into all these details, but look at that 00:55:55.900 --> 00:56:01.700 limitations, we talked about that before. Also, in your research question, in your research 00:56:01.700 --> 00:56:08.700 proposal. There is always something like limitation and that makes sense. You are not improving the 00:56:08.700 --> 00:56:11.400 world. So, do you know your limitations? NOTE Treffsikkerhet: 82% (H?Y) 00:56:11.400 --> 00:56:17.900 Can you generalize it? And is the interpretation correct? It's theinterpretation of the data, is that 00:56:17.900 --> 00:56:25.350 correct? Can we interpret the data actually? Then there are things with RCTs as usual, 00:56:25.350 --> 00:56:30.800 register them on website so that everybody knows what you're doing Etc. But you need to report on 00:56:30.800 --> 00:56:37.900 it. Now, all these details, if you do and we're doing reviews on RCTs, every article is being 00:56:37.900 --> 00:56:39.850 assessed like this. NOTE Treffsikkerhet: 90% (H?Y) 00:56:39.850 --> 00:56:50.100 Again, stop me if you've got a question, I know it's a lot. So that is just an explanation of these 00:56:50.100 --> 00:56:56.200 additional analysis where you can do subgroup analysis Etc. So you've got the overall and then you 00:56:56.200 --> 00:57:02.000 can say ''Well maybe I don't want to look just at the intervention. Maybe I want to look at dosage of 00:57:02.000 --> 00:57:09.500 the intervention. '' That is short and intense, better than long and not that intense. Those could be supper analysis. 00:57:09.500 --> 00:57:26.500 Kine: In randomized controlled studies, how common is it for big studies to not be backed up by RCTs 00:57:26.500 --> 00:57:32.700 or should you expect studies to be backed up by randomized controlled studies? 00:57:32.700 --> 00:57:39.500 Renee: The problem is that we more and more realize that we NOTE Treffsikkerhet: 86% (H?Y) 00:57:39.500 --> 00:57:48.050 need International multicenter trials and we need that because it is very difficult to run an RCT. 00:57:48.050 --> 00:57:52.200 It's costing, you need a lot of money. You need many therapists or clinicians involved or 00:57:52.200 --> 00:57:59.200 educationalists, but it's the best evidence. And what we nowadays have is all quite a few small 00:57:59.200 --> 00:58:05.450 studies. Now, there is a way of combining that, we showed that with the when we talked about reviews, 00:58:05.450 --> 00:58:09.649 that you've got all these small studies and you try with meta-synthesis or NOTE Treffsikkerhet: 90% (H?Y) 00:58:09.649 --> 00:58:16.600 meta-analysis to combine it to have a higher level evidence that way. But of course, the best way is we 00:58:16.600 --> 00:58:26.000 need big RCTs. The bigger the RCTs is also, then you can look at confounders or look at different 00:58:26.000 --> 00:58:32.800 moderators (is a better word). Moderator, meaning it's not necessary confounding because confounding 00:58:32.800 --> 00:58:39.500 is a really a problem, but a moderator can be like, maybe it has NOTE Treffsikkerhet: 80% (H?Y) 00:58:39.500 --> 00:58:44.900 influence on the outcome, maybe age is an influence, then you can do additional analysis if your 00:58:44.900 --> 00:58:51.700 group is big enough. I've got the intervention randomized. I look at pre post. I've compare both 00:58:51.700 --> 00:58:58.700 groups, but maybe I want to say, well actually maybe age is also of influence. Then you look within 00:58:58.700 --> 00:59:03.700 the group, you look at, okay, how are the elderly doing? How is it the bigger your group the more 00:59:03.700 --> 00:59:09.649 moderators you can put in. So I'm not sure that I'm giving a correct answer NOTE Treffsikkerhet: 73% (MEDIUM) 00:59:09.649 --> 00:59:17.200 to you, but we really need bigger studies to take care of moderators to compare really to have 00:59:17.200 --> 00:59:24.900 more sufficient statistical power. If the groups are too small, also not randomized, we don't 00:59:24.900 --> 00:59:31.950 have enough power to say, ''actually, this is statistically significantly working. '' NOTE Treffsikkerhet: 85% (H?Y) 00:59:31.950 --> 00:59:38.000 We can only say there may be a tendency. With a big a RCT you hope that you can actually say 00:59:38.000 --> 00:59:47.300 this is working or not. 00:59:47.300 --> 01:00:29.450 *Kine asking question* 01:00:29.450 --> 01:00:37.800 Usually, if you want to do RCTs, it's almost impossible 01:00:37.800 --> 01:00:42.350 to do it without funding, because it's time-consuming, resource consuming, NOTE Treffsikkerhet: 90% (H?Y) 01:00:42.350 --> 01:00:49.000 it costs a lot of money. It is the best evidence though. So if you go for national brands, 01:00:49.000 --> 01:00:57.000 Norwegian research Council, they really would expect you to do RCTs then. You ask money, you come 01:00:57.000 --> 01:01:02.200 up with a good research design and that is usually in interventions we talk about RCTs, 01:01:02.200 --> 01:01:08.700 correct. So then I expect better. But if you don't have funding is very difficult to do a good one. 01:01:11.500 --> 01:01:18.500 There is another one, the randomized control trial the Rob, the vice Cochran risk of bias. And we 01:01:18.500 --> 01:01:26.400 talked already about the Cochran. Cochran is a database and it's got many reviews there of rcts in 01:01:26.400 --> 01:01:32.700 different areas. We looked at that one. You can go to their website, and they had also a Cat, a 01:01:32.700 --> 01:01:38.200 critical appraisal tool for randomized control trials. And that is the Rob. Now, we've done that 01:01:38.200 --> 01:01:42.399 with four students last year, two pairs of four group and NOTE Treffsikkerhet: 84% (H?Y) 01:01:42.399 --> 01:01:49.200 it's complex, time-consuming, but it is very likely the best cat for RCTs. And since we wanted 01:01:49.200 --> 01:01:56.650 to publish, we did the painful thing and we did the Rob, but it was a bit painful. What Rob does is 01:01:56.650 --> 01:02:04.000 this is a picture that it gives you four different... It is about 01:02:04.000 --> 01:02:09.600 risk of bias in included studies, and it does that for overall, but also for incident 01:02:09.600 --> 01:02:12.250 reporting of results, the outcome, NOTE Treffsikkerhet: 81% (H?Y) 01:02:12.250 --> 01:02:20.900 missing data. How did the researcher, authors dealt with missing data? And we 01:02:20.900 --> 01:02:27.800 all have got missing data because people disappear Etc. Did things change from the intervention, 01:02:27.800 --> 01:02:33.800 did you intervene? Did something go different than planned? And how about randomization? Now, then 01:02:33.800 --> 01:02:40.700 you can see green is good. Red is bad. That's actually how you read it. So green is overall pretty 01:02:40.700 --> 01:02:42.300 good. There are some concerns, NOTE Treffsikkerhet: 90% (H?Y) 01:02:42.300 --> 01:02:50.400 it's yellow. And high risk is the red. So we did, this is from a review we were doing, a review 01:02:50.400 --> 01:02:55.500 on RCTs. And you can see while most of them are pretty good there's a little bit of... especially the 01:02:55.500 --> 01:03:02.500 red that is problematic there. But this is the overview of all included studies. It also gives you 01:03:02.500 --> 01:03:09.550 this paper, this one and this, --actually this table goes on because it got only goes from A to D, 01:03:09.550 --> 01:03:12.200 but I thought you don't need E to Z-- NOTE Treffsikkerhet: 84% (H?Y) 01:03:12.200 --> 01:03:18.600 but there are a lot more studies and for each individual study it gives you the same criteria. And as you can see 01:03:18.600 --> 01:03:25.700 okay, this study, this one has gotten overall problem. There's a high risk. And so you can see 01:03:25.700 --> 01:03:32.400 these are pretty good because you can see all these items are scored. Green is good. But here 01:03:32.400 --> 01:03:37.900 are some doubtful things already, but red is really the problem. So you get a nice overview for your 01:03:37.900 --> 01:03:42.399 paper. And you can also add the individual studies. But again, NOTE Treffsikkerhet: 83% (H?Y) 01:03:42.399 --> 01:03:48.500 Rob is not easy and the students did not like it. Neither did I. But it was a good one. It was a 01:03:48.500 --> 01:03:51.200 good one and you can't always avoid. NOTE Treffsikkerhet: 86% (H?Y) 01:03:52.600 --> 01:03:59.800 Reviews, hang in there. I know it's a little bit boring but hang in. Reviews, if you are 01:03:59.800 --> 01:04:06.900 willing, if you are considering performing a systematic review, you need to discuss Prisma, 01:04:06.900 --> 01:04:12.700 Prisma stands for preferred reporting items for systematic reviews and meta-analysis. If you intend 01:04:12.700 --> 01:04:18.900 to publish and you do a master thesis by publication, you cannot avoid Prisma. It's as simple as 01:04:18.900 --> 01:04:22.000 that. There is an updated Prisma 2020 NOTE Treffsikkerhet: 83% (H?Y) 01:04:22.000 --> 01:04:27.550 and I forgot to update this, but we did it in the other hand out. So you've got the updated list, 01:04:27.550 --> 01:04:33.000 but this is the checklist. We talked about when we talked about literature reviews and 01:04:33.000 --> 01:04:39.100 you can see the title, The Abstract, the introduction, the methods. We've seen 01:04:39.100 --> 01:04:45.100 that by now, and then results, discussion, funding, Etc. You need to record every paper in your 01:04:45.100 --> 01:04:51.800 systematic review. When you perform, you need to report on this, so this is a NOTE Treffsikkerhet: 85% (H?Y) 01:04:51.800 --> 01:05:00.850 a checklist for writing a review, to see the methodological quality of a systematic review. No way 01:05:00.850 --> 01:05:08.200 avoiding Prisma if you do a systematic review and even if you do a scoping review, then nowadays, 01:05:08.200 --> 01:05:14.000 they also want you to do systematic searches. So you still are hanging with Prisma. It's just not 01:05:14.000 --> 01:05:21.000 the whole checklist. There's a different checklist for that. Okay, then there is such a thing as 01:05:21.000 --> 01:05:21.800 diagnostic studies. NOTE Treffsikkerhet: 89% (H?Y) 01:05:21.800 --> 01:05:28.000 And that is, we will have two lectures on diagnostic accuracy and that's got to do with 01:05:28.000 --> 01:05:36.300 sensitivity, specificity and things like that. So you've got a tool and you want to know how good is 01:05:36.300 --> 01:05:44.900 my tool with diagnosing ASD or diagnosing social problems. So that is got to with diagnostic 01:05:44.900 --> 01:05:51.700 accuracy. It's got to do with sensitivity, specificity NOTE Treffsikkerhet: 90% (H?Y) 01:05:51.700 --> 01:05:58.000 and we will talk about that later. And if you are looking at studies like that, then you've got 01:05:58.000 --> 01:06:03.300 also there, you've got another Cat, which is called QUADAS 2, which implies there must be a QUADAS 1, 01:06:03.300 --> 01:06:08.600 and it's a quality assessment tool for diagnostic accuracy studies. Again, 01:06:08.600 --> 01:06:15.700 there's a website and this is again, the whole thing. Now, this is the same, but it's 01:06:15.700 --> 01:06:20.700 too small so I made a little bit bigger. So it is about on the left you can see description, 01:06:20.700 --> 01:06:21.700 signaling questions, NOTE Treffsikkerhet: 77% (H?Y) 01:06:21.700 --> 01:06:29.400 risk of bias, Etc. And on top you can see patient selection, index tests, Etc. Because when you look at 01:06:29.400 --> 01:06:35.399 diagnostics -and again, we will talk about it- and you want to know whether your tool is a good one, 01:06:35.399 --> 01:06:42.350 you need to compare with something like a reference. Like if you develop a new screening tool 01:06:42.350 --> 01:06:48.500 and you want to know how fantastic that screening tool works in diagnosing ADHD, then you want to 01:06:48.500 --> 01:06:51.750 compare it with the gold standard screening tool or whatever NOTE Treffsikkerhet: 75% (MEDIUM) 01:06:51.750 --> 01:06:57.900 is out there and compare it and you can see how good your tool is 01:06:57.900 --> 01:07:02.250 actually working. So they're talking about, that's what they mean with index test and 01:07:02.250 --> 01:07:08.400 reference standard. We will talk about that later. But again, different study designs, different 01:07:08.400 --> 01:07:15.100 Cats are needed. You get beautiful pictures, this is what some people do or this is that you can 01:07:15.100 --> 01:07:20.400 see green is good and red is usually bad. In this case has orange, so you can all 01:07:20.400 --> 01:07:21.800 either do all these beautiful NOTE Treffsikkerhet: 75% (MEDIUM) 01:07:21.800 --> 01:07:29.800 smileys or mon smileys but also a little bit like Rob to you can get here the overall picture for 01:07:29.800 --> 01:07:37.850 all these studies and again green is good, so there is a legenda with that, not going to 01:07:37.850 --> 01:07:46.700 get more into detail here. We've got hours for diagnostic performance anyhow. Now this is from the 01:07:46.700 --> 01:07:51.750 Cochrane and I'm just checking on time. NOTE Treffsikkerhet: 81% (H?Y) 01:07:51.750 --> 01:07:57.500 Cochrane has got a simple one. We did diagnostic reviews, meaning, we've retrieved 01:07:57.500 --> 01:08:04.500 from the literature, all screening tools that were looking for swallowing problems. And we needed to 01:08:04.500 --> 01:08:10.900 know for each article if it is a good one or not. At that time the Cochrane had a very simple one, nine 01:08:10.900 --> 01:08:16.600 items. And you can see this, the first seven had to do with validity of my tool, blind 01:08:16.600 --> 01:08:21.500 interpretation of reference test and index, says to the one test compared to the other one. NOTE Treffsikkerhet: 79% (H?Y) 01:08:21.500 --> 01:08:28.300 Did you use the index test on the reference desk for all participants? It's always got to 01:08:28.300 --> 01:08:34.799 do with validity, study selection, participant selection. Then we had a little bit about 01:08:34.799 --> 01:08:42.300 generalizability. Meaning, can you generalize this study population? If you talk about a screening 01:08:42.300 --> 01:08:50.600 tool in patients with neurological swallowing problems, can I generalize that one to also 01:08:50.600 --> 01:08:51.750 oncological problems? NOTE Treffsikkerhet: 75% (MEDIUM) 01:08:51.750 --> 01:08:57.700 Or is this really just for neurology? So it's about generalizebility. If you cannot 01:08:57.700 --> 01:09:04.000 generalize, it doesn't necessarily need to be bad, but it is a restriction. So you can't use it in 01:09:04.000 --> 01:09:08.899 other groups. But if you can only use it for instance in neurology, that's still a big 01:09:08.899 --> 01:09:15.300 group. But if you can only use it in neurology patients, that are NOTE Treffsikkerhet: 89% (H?Y) 01:09:15.300 --> 01:09:20.899 five years old, that is not generalizeable anymore. Then you've got a problem, that's too 01:09:20.899 --> 01:09:30.100 precise. Last bit has got to do with reliability. So can you reproduce the data? Did they also 01:09:30.100 --> 01:09:37.500 define normal versus or abnormal? Screenings, and we will talk about it. Screening test is not just 01:09:37.500 --> 01:09:44.899 an evaluation. It is actually to identify those at risk. Now, if I've got a screening test, NOTE Treffsikkerhet: 91% (H?Y) 01:09:44.899 --> 01:09:52.399 I need to know when do I decide someone has got an abnormal score is at risk. So, with diagnostic 01:09:52.399 --> 01:10:00.300 tools you want to know, did they give us good enough definitions of what is... if you talk 01:10:00.300 --> 01:10:06.200 about ASD or ADHD or anything, very often there are a number of criteria as you've got these 01:10:06.200 --> 01:10:13.500 cut holes and especially with this screen if you only want to know ''Well, is there a risk of...? 01:10:13.500 --> 01:10:15.100 Are they at risk? '' So you don't NOTE Treffsikkerhet: 71% (MEDIUM) 01:10:15.100 --> 01:10:22.500 diagnose yet, but you ask are they at risk of autism spectrum disorders? You need to make sure that 01:10:22.500 --> 01:10:27.000 they defined what is normal and abnormal according to the authors. NOTE Treffsikkerhet: 75% (MEDIUM) 01:10:27.100 --> 01:10:36.800 Okay. Now the older Cats that I discussed so far, we're only focusing at one time particular type 01:10:36.800 --> 01:10:43.200 of study designs. Sometimes you do a review and you've got all these different study designs in your review 01:10:43.200 --> 01:10:51.800 and you don't want to do for study by Johnson, you use the Rob 2, and for the other one 01:10:51.800 --> 01:10:56.850 you use the Qualsyst... In that case actually NOTE Treffsikkerhet: 91% (H?Y) 01:10:56.850 --> 01:11:04.050 I would prefer to have one cat that can deal with different study designs. And Qualsyst by Kmet 01:11:04.050 --> 01:11:11.900 is one of those study or Cats that can include different types of study designs. It's about 01:11:11.900 --> 01:11:19.300 criteria. *Kine asking question* 01:11:19.300 --> 01:11:35.450 *Kine asking question* 01:11:35.450 --> 01:11:42.300 Renee: All that we discuss is critical appraisal tools. I call that short Cats. 01:11:48.750 --> 01:11:56.700 And this one, this Cat is called Qualsyst and Kmet is the name of the first author and the NOTE Treffsikkerhet: 74% (MEDIUM) 01:11:56.700 --> 01:12:03.300 advantage of this cat is that it allows different study designs being rated by the same Cat which is 01:12:03.300 --> 01:12:10.600 easier and it comes with an overall score and not all Cats do that. Sometimes you use the Cat and 01:12:10.600 --> 01:12:17.900 then you've got 30 individual items. And then what? So this one is gives you a kind of overall 01:12:17.900 --> 01:12:24.800 rating. And this is how it looks like, you've got here all these criteria and you can say, yes, 01:12:24.800 --> 01:12:26.850 that's two points, one point, NOTE Treffsikkerhet: 88% (H?Y) 01:12:26.850 --> 01:12:34.100 zero, sometimes not applicable. For instance, item 5 is about random allocation. Well, if 01:12:34.100 --> 01:12:40.000 we're talking about a case series, there's no random allocation at all. So then you just say not 01:12:40.000 --> 01:12:46.900 applicable. So the advantage of courses is something like, you've got different types of study 01:12:46.900 --> 01:12:54.800 designs can be included and you get as kind of overall rating that usually looks like this. So this 01:12:54.800 --> 01:12:56.650 is the score that this NOTE Treffsikkerhet: 86% (H?Y) 01:12:56.650 --> 01:13:05.400 he had 18 out of 22 and that makes 82%. And the 01:13:05.400 --> 01:13:12.700 reason why this is different, these numbers here, is because for instance, the RCT, maybe 01:13:12.700 --> 01:13:18.300 ... and therefore he could have higher more items were applicable. So if the 01:13:18.300 --> 01:13:23.800 items are not applicable, you can't score it. Therefore, you make it into a percentage score. 01:13:23.800 --> 01:13:26.900 So overall these articles in a review, NOTE Treffsikkerhet: 86% (H?Y) 01:13:26.900 --> 01:13:33.600 get a Kmet score and that means we translate that in wording here, we say 01:13:33.600 --> 01:13:39.900 anything above 80% is good. So these two guys are pretty good. This one is strong because he's 01:13:39.900 --> 01:13:48.800 between sixty and seventy nine. And this one is 54% only and that is adequate. Usually if they score poor 01:13:48.800 --> 01:13:54.700 they're out because that's really a dodgy paper. I don't want to include that one. 01:13:54.700 --> 01:13:57.600 I ignore their results. NOTE Treffsikkerhet: 88% (H?Y) 01:13:58.500 --> 01:14:05.300 In short, if you want to know more about critical appraisal tools or cats, you can go to this 01:14:05.300 --> 01:14:11.300 website. You can see all these different things, for instance this is the middle bit. 01:14:11.300 --> 01:14:18.500 If you enlarge that, then you can see here there are all these reporting guidelines per main study 01:14:18.500 --> 01:14:26.000 type. So for randomized trials, observational studies Etc. All these examples behind that there 01:14:26.000 --> 01:14:26.750 are over NOTE Treffsikkerhet: 90% (H?Y) 01:14:26.750 --> 01:14:31.900 400 different cats that they have got there. So you don't want to see all of them. That's why I just 01:14:31.900 --> 01:14:41.400 gave you examples but please go to this website if you want to know more about cats. Now this 01:14:41.400 --> 01:14:48.300 website also gives you kind of a guideline. So that is this one. It's always 01:14:48.300 --> 01:14:53.100 overwhelming I think at the beginning, but this is where they start. So you start on the topic and 01:14:53.100 --> 01:14:56.850 say, okay. Is it on Research? No, they go there. is it on Research on NOTE Treffsikkerhet: 86% (H?Y) 01:14:56.850 --> 01:15:03.700 humans, you go down. Did you have quantitative data? Yes, you got. So a little bit like that you 01:15:03.700 --> 01:15:10.800 end up in a certain place. And they say for instance here, is it a review? And they refer 01:15:10.800 --> 01:15:17.700 you to Prisma. If it is a randomized controlled trial, it refers you to Consort or maybe Strop 01:15:17.700 --> 01:15:26.650 or maybe the Rob 2, that was not yet here yet. So it gives you some help to which one to select. NOTE Treffsikkerhet: 80% (H?Y) 01:15:26.650 --> 01:15:36.400 But what you need to remember, if it is poor study quality, delete it. You can 01:15:36.400 --> 01:15:42.600 report on it but what you say is so you include these articles for instance in a review. You must 01:15:42.600 --> 01:15:48.700 include them. You say these are retrieved, then you do your critical appraisal tool and you say, 01:15:48.700 --> 01:15:55.800 well actually these articles are so poor. The methods is so poor, you've report on that, but from that 01:15:55.800 --> 01:15:57.200 moment on you just report others, NOTE Treffsikkerhet: 82% (H?Y) 01:15:57.200 --> 01:16:02.900 but you do not give any further details from these articles. You just say I've done my Cats, 01:16:02.900 --> 01:16:09.700 this is the outcome. These are too poor to include anywhere further in my studies, 01:16:09.700 --> 01:16:19.700 I'm going to ignore them. Okay, so that was in short bias and confounding, study designs at a glance and 01:16:19.700 --> 01:16:26.500 Cats critical appraisal tools. Are there any questions at this moment? NOTE Treffsikkerhet: 78% (H?Y) 01:16:26.500 --> 01:16:35.900 I'm going to quit. I'm going to stop recording here. I think you don't want to have all the chats now.