Workshop - Evaluating effects of non-pharmacological interventions
-The Randomised Controlled Trial - 19.4. - 20.4.2010
Course supervisor: Kåre Birger Hagen
LOCATION: Lunsjrom 4 et.(Helsefag eller Forebyggende medisin), Gydas vei 8
MONDAY APRIL 19
9.00 – 9.10 Introduction (10 minutes)
9.10 –10.30 Topic 1: Internal validity (80 minutes)
• The logic of randomised trials
• Allocation concealment
• Blinding of patients, therapists, assessors and statisticians
• Maximising follow-up
10.30 – 10.45 Morning tea
10.45 –11.45 Topic 2: Matching the design to the question (60 minutes)
• Explanatory and pragmatic trials
• Control interventions (wait-lists, sham treatments and head-to-head comparisons)
• Design options (pre-test/post-test, frequency of follow-up, multiple arm trials, factorial trials, within-subject comparisons, cross-over trials, cluster-randomised trials)
11.45 –12.45 Practical Session 1: Design (60 minutes)
12.45 – 1.30 Lunch
1.30 – 2.30 Topic 3: Allocation (60 minutes)
• Allocation schedules – theoretical considerations (two views of confounding, unrestricted randomisation, single blocks, multiple blocks, stratification, minimisation)
• Allocation schedules and procedures – practical considerations
2.30 – 3.15 Practical Session 2: Allocation (60 minutes)
3.15 – 3.30 Afternoon tea
3.30 – 4.30 Topic 4: Statistical principles in design (60 minutes)
• Why statistical considerations constrain design
• Minimising Type I errors (primary and secondary comparisons, multiple looks, subgroup analyses)
• Minimising Type II errors
TUESDAY APRIL 20
9.00 – 10.00 Topic 5: Sample size (60 minutes)
• Overview (the logic of sample size calculation)
• Continuous outcomes (two-armed trials, loss to follow-up and non-compliance, unequal allocation ratios, factorial trials, within-subject and cross-over trials, cluster-randomised trials)
• Incidence proportions and incidence rates
• Times to event
10.00 – 11.00 Practical Session 3: Sample size (60 minutes)
11.00 – 11.15 Morning tea
11.15 – 11.45 Topic 6: Choosing outcomes (30 minutes)
• Properties of good outcome measures
• Continuous outcomes (post-test scores, change scores, percentage change scores, covariate-adjusted scores, choosing covariates)
• Incidence proportions and incidence rates
• Times to event
11.45 – 12.30 Topic 7: Introductory analysis, part I (45 minutes)
• Data screening and dealing with missing data
• Analysis by intention to treat
• Principles of analysis (estimation not hypothesis testing, interpreting size of effects)
12.30 – 1.15 Lunch
1.15 – 2.00 Topic 7: Introductory analysis, part II (45 minutes)
• Continuous well-behaved data and badly behaved data
• Incidence proportions
• Times to event
2.00 – 3.00 Practical Session 4: Analysis (60 minutes)
3.00 – 3.30 Topic 8: The smallest worthwhile effects of an intervention (30 minutes)
3.30 – 4.00 Afternoon tea
4.00 – 4.30 Discussion of any outstanding issues (30 minutes)
4.30 – 4.40 Workshop evaluation, close
WORKSHOP OBJECTIVES
At the end of the workshop, participants should be able to:
1. design a randomised trial which incorporates key elements which maximise internal validity
2. match trial designs to particular research questions
3. describe statistical principles that guide the design and analysis of clinical trials
4. recognise constraints on the number and type of outcome to be investigated
5. estimate sample sizes required for a range of simple trial designs
6. express outcomes in a form that maximises precision of estimates of treatment effects
7. use simple methods used to analyse randomised trials (comparisons of continuous outcomes, and incidence proportions, incidence rates and time to events)
8. describe the benefit-harm method for eliciting estimates of the smallest worthwhile effect of intervention
Note on practical sessions:
Most of the sessions contain suggestions for more activities than it will be possible to complete in the allotted times. Each group should make a decision about which activities most interest group members and should complete those activities as a first priority.
It is not necessary for all members of a group to work on the same task. If there is a range of interests there is no reason why the group cannot break into subgroups that tackle different activities.
Rob Herbert
Rob Herbert is NHMRC Senior Research Fellow at The George Institute for International Health, Associate Professor at the Sydney Medical School of the University of Sydney, and Honorary Research Fellow at the Prince of Wales Medical Research Institute. He conducts parallel programs of clinical research investigating the effectiveness of physiotherapy interventions with randomised controlled trials and laboratory research investigating mechanisms of contracture. Rob is a founding Director of the Centre for Evidence-Based Physiotherapy which produces the PEDro database, and was Scientific Editor of the Australian Journal of Physiotherapy from 2001-2005. He is currently a member of the Editorial Board of the Journal of Applied Physiology.
Practical Session 1: Design
You could begin this first practical session by discussing, as a group, the design of clinical trials that group members plan to do, or are doing, or have done. The focus should be on:
• methods of concealment, blinding and follow-up
• the choice between explanatory and pragmatic designs
• the type of control intervention
• the number and timing of measurement occasions
• the benefits or otherwise of factorial designs, or within-subject or cross-over comparisons
• the unit of randomisation
Leave issues of allocation for now, as we will focus on these in another practical session.
Consider design alternatives and justify the chosen design. Debate the alternatives.
NOTES
Practical Session 2: Allocation
You could begin this session by discussing, as a group, the allocation procedures for clinical trials that group members plan to do. Where group members are conducting or have completed a trial you may like to review their allocations procedures. You could draft or revise the section of your protocol which describes the allocation procedures. Other group members could provide feedback.
If you have time, you might want to practice generating an allocation schedule. For example, you could:
1. Generate an allocation schedule for a 2 × 2 factorial trial of 100 participants. Constrain allocation so that the groups are the same size.
2. Cut and paste the allocation into Word for producing allocation sheets to be inserted into envelopes.
3. Generate a simple (unrestricted) random allocation schedule for a three-armed trial with 30 participants.
4. Generate an allocation schedule for a two-armed trial of 60 participants, using random permuted blocks of varying block size.
NOTES
Practical Session 3: Sample size
You could begin this session by discussing, as a group, the sample size requirements for clinical trials that group members plan to do. Where group members are conducting or have completed a trial you may like to review their justifications of sample size. You could draft or revise the section of your protocol which justifies the sample size – ensure you provide enough information to permit replication of your calculations. Other group members could provide feedback.
If you have time, you might want to answer the following questions:
1. How many subjects would you need for a trial designed to investigate the effects of 6 week program of stretching on muscle soreness after exercise? Soreness is measured on a 10-point scale (0 = no soreness, 10 = worst soreness I have ever had.) Assume that published data show SDs of soreness measures in a comparable population are about 2.0 prior to intervention, and 1.5 after a 6 week intervention.
2. How many subjects would you need for a trial designed to investigate the effects of stretching on injury risk? Assume that the risk of injury in one year in people who do not stretch is 15%. Compare the sample size you would need if the outcome was expressed as time to injury.
If you are interested in trials which require other sorts of sample size analyses (for example, a trial with time to event data, or a cluster-randomised trial, or a factorial trial in which the interaction is of interest) you could estimate the required sample sizes for those trials.
NOTES
Practical Session 4: Analysis
You could begin this session by discussing, as a group, the analysis for clinical trials that group members plan to do. Where group members are conducting or have completed a trial you may like to review their analysis strategies. Consider how outcomes are expressed and how the inferential analyses are implemented. You could draft or revise the section of your protocol which describes the analysis. Other group members could provide feedback.
If you have get time, you might want to practice doing some analyses. Perform the following analyses:
(a) The Knee range.xls file provides fictitious data from an imaginary trial of the effects of exercise after knee arthroplasty on knee range of motion. Are these outcome data well-behaved? Estimate the size of the effect of exercise, with 95% CIs, using the Confidence Interval Calculator to estimate the difference in means.
(b) The Respiratory complications.xls file provides fictitious data from an imaginary trial of the effects of postural drainage on incidence of respiratory complications. Estimate the size of the effect of postural drainage on the risk of respiratory complications, with 95% CIs, using the Confidence Interval Calculator.
NOTES
SELECTED REFERENCES
Textbooks on clinical trials
Friedman LM, Furberg CD, DeMets DL. Fundamentals of Clinical Trials. New York: Springer, 1998. [I]
Pocock SJ. Clinical Trials: A Practical Approach. Chichester: Wiley, 1984. [I]
Textbooks on statistical analysis of clinical trials
Piantadosi S. Clinical trials: A Methodologic Perspective. Hoboken: Wiley-Interscience, 2005.
Senn S. Statistical Issues in Drug Development. Chichester: Wiley, 1997.
Design
Donner A, Klar N. Design and Analysis of Cluster Randomization Trials in Health Research. London: Arnold, 2000.
Herbert R. Explanatory and pragmatic clinical trials. In: Gad SC, editor. Clinical Trials Handbook: Wiley; 2009. p. 1087-1104.
Senn S. Cross-over Trials in Clinical Research. Chichester: Wiley, 1993.
Sample size calculation
Friedman LM, Furberg CD, DeMets DL. Fundamentals of Clinical Trials. New York: Springer, 1998. [I]
Julious SA (2004) Sample sizes for clinical trials with normal data. Statistics in Medicine 23: 1921-1986.
Sahai H, Khurshid A (1996) Formulae and tables for the determination of sample sizes and power in clinical trials for testing differences in proportions for the two-sample design: a review. Statistics in Medicine 15: 1-21.
Interactions and subgroup analyses
Brookes ST, Whitely E, Egger M, Smith GD, Mulheran PA, Peters TJ (2004) Subgroup analyses in randomized trials: risks of subgroup-specific analyses; power and sample size for the interaction test. Journal of Clinical Epidemiology 57: 229-236.
Allocation and concealment
Berger VW. Selection Bias and Covariate Imbalances in Clinical Trials. Hoboken, Wiley, 2005.
Scott NW, McPherson GC, Ramsay CR, Campbell MK (2002) The method of minimization for allocation to clinical trials. A review. Controlled Clinical Trials 23: 662-674.
Statistical inference
Barnett V. Comparative Statistical Inference. Chichester: Wiley, 1999. [A]
Rothman KJ, Greenland S. Modern Epidemiology (2nd edn). Philadelphia: Lippincott, 1998, Chapter 12.
Royall RM (1997). Statistical Evidence: A Likelihood Paradigm. Boca Raton: Chapman & Hall, 1997. [A]
Statistical Methods
Altman DG, Gardner MJ. Statistics with Confidence: Confidence Intervals and Statistical Guidelines. 2nd ed. London: BMJ, 2000. [I]
Carpenter J, Bithell J (2000) Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticians. Statistics in Medicine 19: 1141-1164.
Cleves MA, Gould WW, Gutierrez RG, Marchenko YU. An Introduction to Survival Analysis Using Stata. College Station, Texas: Stata Press, 2008.
Glantz SA. Primer of Biostatistics. 6th ed. New York: McGraw-Hill Professional, 2005. [I]
Long JS, Freese J. Regression Models for Categorical Dependent Variables Using Stata. 2nd ed. College Station, Tex.: Stata Press, 2006.
Newcombe RG (1998) Interval estimation for the difference between independent proportions: comparison of eleven methods. Statistics in Medicine 17: 873-890.
Twisk JWR. Applied Longitudinal Data Analysis for Epidemiology: A Practical Guide. Cambridge: Cambridge University Press, 2003.
[I] = introductory
[A] = advanced