Abstract
Using the best quality of clinical research evidence is essential for choosing the
right treatment for patients. How to identify the best research evidence is, however,
difficult. In this narrative review we summarise these threats and describe how to
minimise them. Pertinent literature was considered through literature searches combined
with personal files. Treatments should generally not be chosen based only on evidence
from observational studies or single randomised clinical trials. Systematic reviews
with meta-analysis of all identifiable randomised clinical trials with Grading of
Recommendations Assessment, Development and Evaluation (GRADE) assessment represent
the highest level of evidence. Even though systematic reviews are trust worthier than
other types of evidence, all levels of the evidence hierarchy are under threats from
systematic errors (bias); design errors (abuse of surrogate outcomes, composite outcomes,
etc.); and random errors (play of chance). Clinical research infrastructures may help
in providing larger and better conducted trials. Trial Sequential Analysis may help
in deciding when there is sufficient evidence in meta-analyses. If threats to the
validity of clinical research are carefully considered and minimised, research results
will be more valid and this will benefit patients and heath care systems.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to European Journal of Internal MedicineAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- The Royal College of Physicians of Edinburgh: James Lind Library.([Available online at])http://wwwjameslindlibraryorg/Date: 2003
- Commentary: the evolution of methods to assess the effects of treatments, illustrated by the development of treatments for diphtheria, 1825–1918.Int J Epidemiol. 2013; 42: 662-676
- Studier over den statistiske undersøgelsesmetode som hjælpemiddel ved terapeutiske undersøgelser [Studies on the statistical study design as an aid in therapeutic trials] (http://www.jameslindlibrary.org/illustrating/records/studier-over-den-statistiske-undersogelsesmetode-som-hjaelpemid/key_passages).Bibl Laeger. 1897; 89: 1-40
- Likely country of origin in publications on randomised controlled trials and controlled clinical trials during the last 60 years.Trials. 2007; 8: 7
- The Cochrane Library.([Avaiable at])
- Description of the project.([Available from])
Durisic S, Garattini S, Rath A, Neugebauer EAM, Laville M, Jakobsen JC, Kubiac C, DeMotes-Mainard J, Gluud C: Common barriers to the conduct of randomised clinical trials - the Europeean Clinical Research Infrastructure (ECRIN) perspective Trials, 2016 [to be submitted].
Rath A, Ngwabyt S, Durisic S, Garattini S, Neugebauer EAM, Laville M, Jakobsen JC, Kubiac C, DeMotes-Mainard J, Gluud C: Specific barriers to the conduct of randomised clinical trials within rare diseases — the Europeean Clinical Research Infrastructure (ECRIN) perspective Trials, 2016 [to be submitted].
Neugebauer EAM, Rath A, Durisic S, Garattini S, Laville M, Jakobsen JC, Kubiac C, DeMotes-Mainard J, Gluud C: Specific barriers to the conduct of randomised clinical trials on medical devices — the Europeean Clinical Research Infrastructure (ECRIN) perspective Trials, 2016 [to be submitted].
Laville M, Neugebauer EAM, Rath A, Durisic S, Garattini S, Jakobsen JC, Kubiac C, DeMotes-Mainard J, Gluud C: Specific barriers to the conduct of randomised clinical trials on nutrition - the Europeean Clinical Research Infrastructure (ECRIN) perspective Trials, 2016 [to be submitted].
- The Cochrane Handbook for Systematic Reviews of Interventions, Version 5.1.0. The Cochrane Collaboration.([Available from])Date: 2011
- Evidence at a glance: error matrix approach for overviewing available evidence.BMC Med Res Methodol. 2010; 10: 90
- The necessity of randomized clinical trials.Br J Med Res. 2013; 3: 1453-1468
- Evaluating non-randomised intervention studies.Health Technol Assess. 2003; 7 ([iii-x]): 1-173
- The number of patients and events required to limit the risk of overestimation of intervention effects in meta-analysis-a simulation study.PLoS One. 2011; 6e25491
- Hepatology may have problems with putative surrogate outcome measures.J Hepatol. 2007; 46: 734-742
- Increasing value and reducing waste in biomedical research regulation and management.Lancet. 2014; 383: 176-185
- How to increase value and reduce waste when research priorities are set.Lancet. 2014; 383: 156-165
- Avoidable waste in the production and reporting of research evidence.Lancet. 2009; 374: 86-89
- Reducing waste from incomplete or unusable reports of biomedical research.Lancet. 2014; 383: 267-276
- Biomedical research: increasing value, reducing waste.Lancet. 2014; 383: 101-104
- Increasing value and reducing waste in biomedical research: who's listening?.Lancet. 2016; 387: 1573-1586
- How to make more published research true.PLoS Med. 2014; 11e1001747
- Clinical trials: what a waste.BMJ. 2014; 349: g7089
- Increasing value and reducing waste in research design, conduct, and analysis.Lancet. 2014; 383: 166-175
- Why most published research findings are false.PLoS Med. 2005; 2e124
- How to read clinical journals.Can Med Assoc J. 1982; 126: 1373
- Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs.Syst Rev. 2006; 333
- The thresholds for statistical and clinical significance — a five-step procedure for evaluation of intervention effects in randomised clinical trials.BMC Med Res Methodol. 2014; 14
- Thresholds for statistical and clinical significance in systematic reviews with meta-analytic methods.BMC Med Res Methodol. 2014; 14: 120
- Discordance between meta-analyses and large-scale randomized, controlled trials. Examples from the management of acute myocardial infarction.Ann Intern Med. 1995; 123: 873-877
- The need for large-scale randomized evidence without undue emphasis on small trials, meta-analyses, or subgroup analyses.JAMA. 2009; 302: 2361-2362
- Is meta-analysis the platinum standard of evidence?.Stud Hist Philos Biol Biomed Sci. 2011; 42: 497-507
- Intersection of systematic review methodology with the NIH reproducibility initiative.Environ Health Perspect. 2014; 122: A176-A177
- Obtaining evidence by a single well-powered trial or several modestly powered trials.Stat Methods Med Res. 2016; 25: 538-552
- Demystifying trial networks and network meta-analysis.BMJ. 2013; 346: f2914
- Efficacy of new generation antidepressants: differences seem illusory.PLoS One. 2013; 8e63509
- Impact of reporting bias in network meta-analysis of antidepressant placebo-controlled trials.PLoS One. 2012; 7e35219
- The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations.Ann Intern Med. 2015; 162: 777-784
- Preferred reporting items for a systematic review and meta-analysis of individual participant data: the PRISMA-IPD statement.JAMA. 2015; 313: 1657-1665
- Comparison of evidence of treatment effects in randomized and nonrandomized studies.JAMA. 2001; 286: 821-830
- Agreement of treatment effects for mortality from routinely collected data and subsequent randomized trials: meta-epidemiological survey.BMJ. 2016; 352: i493
- Randomize the first patient.N Engl J of Med. 1977; 296: 107
- Are mortality differences detected by administrative data reliable and actionable?.JAMA. 2013; 309: 1410-1411
- Statistical Development of Quality in Medicine.Wiley, 2007
- Influence of reported study design characteristics on intervention effect estimates from randomized controlled trials: combined analysis of meta-epidemiologic studies.Health Technol Assess. 2012; 16: 1-82
- Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study.BMJ. 2008; 336: 601-605
- Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials.Ann Intern Med. 2012; 157: 429-438
- The prevention and treatment of missing data in clinical trials.N Engl J Med. 2012; 367: 1355-1360
- Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls.BMJ. 2009; 338
- Systematic review of the empirical evidence of study publication bias and outcome reporting bias.PLoS One. 2008; 3e3081
- Reporting bias group: systematic review of the empirical evidence of study publication bias and outcome reporting bias — an updated review.PLoS One. 2013; 8e66844
- Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles.JAMA. 2004; 291: 2457-2465
- The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration.BMJ. 2009; 339: b2700
- Transaprency and registration in clinical research in the Nordic countries.Nordic Trial Alliance, NordForsk, 2015: 1-108
- Industry sponsorship and research outcome.Coch Database Syst Rev. 2012; 12MR000033
- Developing core outcome sets for clinical trials: issues to consider.Trials. 2012; 13: 132
- Driving up the quality and relevance of research through the use of agreed core outcomes.J Health Serv Res Policy. 2012; 17: 1-2
- Patient involvement in health research: a contribution to a systematic review on the effectiveness of treatments for degenerative ataxias.Soc Sci Med. 2009; 69: 920-925
- Identifying the clinical domains of fibromyalgia: contributions from clinician and patient Delphi exercises.Arthritis Rheum. 2008; 59: 952-960
- Composite outcomes in randomized trials - greater precision but with greater uncertainty?.JAMA. 2003; 289: 2554-2559
- Definition, reporting, and interpretation of composite outcomes in clinical trials: systematic review.BMJ. 2010; 341: c3920
- ICH E9 guideline ‘statistical principles for clinical trials’: a case study.Stat Med. 2003; 22 ([discussion 13-17]): 1-11
- Non-inferiority trials are unethical because they disregard patients' interests.Lancet. 2007; 370: 1875-1877
- Dissemination and publication of research findings: an updated review of related biases.Health Technol Assess. 2010; 14 ([ix-xi, 1-193]): iii
- All Trials Registered, All Results Reported.([avaiable at])
- Relation of completeness of reporting of health research to journals' endorsement of reporting guidelines: systematic review.BMJ. 2014; 348: g3804
- Some main problems eroding the credibility and relevance of randomized trials.Bull NYU Hosp Jt Dis. 2008; 66: 135-139
- Clinical trials should begin and end with systematic reviews of relevant evidence: 12 years and waiting.Lancet. 2010; 376: 20-21
- Bringing it all together: Lancet-Cochrane collaborate on systematic reviews.Lancet. 2001; 357: 1728
- Putting clinical trials into context.Lancet. 2005; 366: 107-108
- on the Community code relating to medicinal products for human use (Consolidated version: 16/11/2012).6 November 2001
- How can we regulate medicines better?.BMJ. 2007; 335: 803-805
- The scientific community should lobby to be able to apply for drug licences.BMJ. 2012; 344e3553
- Good Pharma.Palgrave Macmillan, 2015
- Honorary and ghost authorship in high impact biomedical journals: a cross sectional survey.BMJ. 2011; 343: d6128
- Berlin JA et al: SPIRIT 2013 statement: defining standard protocol items for clinical trials.Ann Intern Med. 2013; 158: 200-207
- CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials.Ann Int Med. 2010; 152: 726-732
- Sample size calculations in clinical research.2nd ed. Chapman & Hall/CRC Biostatistics Series. Chapman and Hall/CRC, 2007
- Interim analysis: the alpha spending function approach.Stat Med. 1994; 13: 1341-1356
- A statistical paradox.Biometrika. 1957; 44: 187-192
- Repeated confidence intervals for group sequential clinical trials.Control Clin Trials. 1984; 5: 33-45
- Trial sequential analysis reveals insufficient information size and potentially false positive results in many meta-analysis.J Clin Epidemiol. 2008; 61: 763-769
- User manual for trial sequential analysis (TSA).Copenhagen Trial Unit, Centre for Clinical Intervention Research, Copenhagen, Denmark2011: 1-115 ([Available from http://www.ctu.dk/tsa])
- Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis.J Clin Epidemiol. 2008; 61: 64-75
- The impact of study size on meta-analyses: examination of underpowered studies in Cochrane reviews.PLoS One. 2013; 8e59202
- The knowledge system underpinning healthcare is not fit for purpose and must change.BMJ. 2015; 350: h2463
- Estimating required information size by quantifying diversity in random-effects model meta-analyses.BMC Med Res Methodol. 2009; 9: 86
- Re: the knowledge system underpinning healthcare is not fit for purpose and must change — reponses to the opposing viewpoints of Roberts and colleagues and Tovey and colleagues.BMJ. 2015;
- Sequential methods for random-effects meta-analysis.Stat Med. 2011; 30: 903-921
- Discrete sequential boundaries for clinical trials.Biometrika. 1983; 70: 659-663
- Trial sequential analysis reveals insufficient information size and potentially false positive results in many meta-analysis.J Clin Epidemiol. 2008; 61: 763-769
- Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses.Lancet. 1999; 354: 1896-1900
- Bias in meta-analysis detected by a simple, graphical test.BMJ. 1997; 315: 629-634
- Operating characteristics of a rank correlation test for publication bias.Biometrics. 1994; 50: 1088-1101
- Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis.Biometrics. 2000; 56: 455-463
- Empirical assessment of effect of publication bias on meta-analyses.BMJ. 2000; 320: 1574-1577
- Epidemiology and reporting characteristics of systematic reviews.PLoS Med. 2007; 4e78
- Systematic review finds overlapping reviews were not mentioned in every other overview.J Clin Epidemiol. 2014; 67: 368-375
- Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study.BMJ. 2005; 330: 1053
- Antioxidant supplements for prevention of mortality in healthy participants and patients with various diseases.Cochrane Database Syst Rev. 2012; 3CD007176
- A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis.BMJ. 2014; 349: g5630
- New developments in the conduct and management of multi-center trials: an international review of clinical trial units.Fundam Clin Pharmacol. 1995; 9: 284-289
- No surgical innovation without evaluation: the IDEAL recommendations.Lancet. 2009; 374: 1105-1112
Article info
Publication history
Published online: May 07, 2016
Accepted:
March 22,
2016
Received:
March 15,
2016
Identification
Copyright
© 2016 European Federation of Internal Medicine. Published by Elsevier B.V. All rights reserved.
