Advertisement

Assessment of frequency and reporting of design changes among clinical drug trials published in influential medical journals

  • Daniel Shepshelovich
    Affiliations
    Medicine A, Rabin Medical Center, Beilinson Hospital, Jabotinsky 39, Petah Tikva 4941492, Israel

    Sackler School of Medicine, Tel-Aviv University, Ramat Aviv, Tel Aviv 6997801, Israel
    Search for articles by this author
  • Dafna Yahav
    Affiliations
    Sackler School of Medicine, Tel-Aviv University, Ramat Aviv, Tel Aviv 6997801, Israel

    Infectious Diseases Unit, Rabin Medical Center, Beilinson Hospital, Jabotinsky 39, Petah Tikva 4941492, Israel
    Search for articles by this author
  • Ariadna Tibau
    Affiliations
    Oncology Department, Hospital de la Santa Creu i Sant Pau, Universitat Autònoma de Barcelona, Carrer se Dant Quinti 89, Barcelona 08041, Spain
    Search for articles by this author
  • Eitan Amir
    Correspondence
    Corresponding author.
    Affiliations
    Division of Medical Oncology & Hematology, Department of Medicine, Princess Margaret Cancer Centre, The University of Toronto, 610 University Ave, Toronto, Ontario M5G 2M9, Canada
    Search for articles by this author
Published:August 19, 2019DOI:https://doi.org/10.1016/j.ejim.2019.08.009

      Highlights

      • Modifications of trial design after initiation can undermine scientific validity.
      • Design changes were identified in most drug trials published in influential journals.
      • Most modifications were not reported in the related publications.
      • Justification for such modifications should be reported more clearly.

      Abstract

      Background

      Modifications of primary outcome measures after trial initiation can undermine their scientific validity. We aimed to quantify these changes and to characterize potential predictors in clinical drug trials published in high impact factor general medicine journals.

      Methods

      The study cohort included all prospective, adult drug clinical trials published in the New England Journal of Medicine, JAMA and Lancet between June 2017 and May 2018. The matching ClinicalTrials.gov entries effective at trial initiation were compared with those effective on July 2018, thereby identifying amendments to primary outcome definitions and assessment timeframes and the planned sample size (defined as >10% change). The primary publications were reviewed for reporting of these amendments. Associations between identified changes and trial characteristics were explored using logistic regression.

      Results

      Of the 147 included trials, modifications to primary outcome measures were identified in 80 (54%). Primary outcome definitions, outcome assessment timeframes and the planned sample size were modified in 28 (19%), 12 (8%) and 65 (45%) of registry entries, respectively; of which 21 (75%), 11 (92%), and 33 (51%), respectively, were not reported in the related publications. There were no significant associations between modifications in registry entries and study characteristics.

      Conclusions

      Approximately half of trials published in influential medical journals present changes to the design while patient accrual is ongoing, and two thirds of them are unreported. Justification for such modifications should be reported more clearly. Reviewers, editors and readers should consult ClinicalTrials.gov for a more comprehensive report of the evolution of key study methods.

      Abbreviations:

      CI (confidence intervals), HR (hazard ratio), IQR (interquartile range), FDA (Food and Drug Administration), OR (odds ratios), RCT (randomized controlled trial)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to European Journal of Internal Medicine
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Chan A.W.
        • Hróbjartsson A.
        • Haahr M.T.
        • Gøtzsche P.C.
        • Altman D.G.
        Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles.
        JAMA. 2004 May 26; 291: 2457-2465
        • Dwan K.
        • Gamble C.
        • Williamson P.R.
        • Kirkham J.J.
        • Reporting Bias Group
        Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review.
        PLoS One. 2013 Jul 5; 8e66844
        • Chan A.W.
        • Pello A.
        • Kitchen J.
        • et al.
        Association of trial registration with reporting of primary outcomes in protocols and publications.
        JAMA. 2017 Nov 7; 318: 1709-1711
        • Mathieu S.
        • Boutron I.
        • Moher D.
        • Altman D.G.
        • Ravaud P.
        Comparison of registered and published primary outcomes in randomized controlled trials.
        JAMA. 2009 Sep 2; 302: 977-984
        • De Angelis C.
        • Drazen J.M.
        • Frizelle F.A.
        • et al.
        Clinical trial registration: a statement from the International Committee of Medical Journal Editors.
        N Engl J Med. 2004; 351: 1250-1251
        • U.S. Public Law 110-85
        • Zarin D.A.
        • Tse T.
        Trust but verify: trial registration and determining fidelity to the protocol.
        Ann Intern Med. 2013; 159: 65-67
        • Zarin D.A.
        • Tse T.
        • Williams R.J.
        • Rajakannan T.
        Update on trial registration 11 years after the ICMJE policy was established.
        N Engl J Med. 2017; 376: 383-391
        • Shepshelovich D.
        • Tibau A.
        • Molto C.
        • et al.
        Assessment of frequency and reporting of changes in Cancer trial design after initiation of patient accrual.
        JAMA Oncol. 2018 Dec 6; ([Epub ahead of print])https://doi.org/10.1001/jamaoncol.2018.5877
        • Liang F.
        • Guo X.
        • Zhang S.
        • Xue H.
        • Chen Q.
        • Hu X.
        Comparison of primary endpoints between publications, registries, and protocols of phase III cancer clinical trials.
        Oncotarget. 2017 Oct 3; 8: 97648-97656
        • Raghav K.P.
        • Mahajan S.
        • Yao J.C.
        • et al.
        From protocols to publications: a study in selective reporting of outcomes in randomized trials in oncology.
        J Clin Oncol. 2015 Nov 1; 33: 3583-3590
        • Perlmutter A.S.
        • Tran V.T.
        • Dechartres A.
        • Ravaud P.
        Statistical controversies in clinical research: comparison of primary outcomes in protocols, public clinical-trial registries and publications: the example of oncology trials.
        Ann Oncol. 2017 Apr 1; 28: 688-695
        • Hartung D.M.
        • Zarin D.A.
        • Guise J.M.
        • McDonagh M.
        • Paynter R.
        • Helfand M.
        Reporting discrepancies between the ClinicalTrials.gov results database and peer-reviewed publications.
        Ann Intern Med. 2014 Apr 1; 160: 477-483
        • Jones C.W.
        • Keil L.G.
        • Holland W.C.
        • Caughey M.C.
        • Platts-Mills T.F.
        Comparison of registered and published outcomes in randomized controlled trials: a systematic review.
        BMC Med. 2015 Nov 18; 13: 282
        • Ramagopalan S.
        • Skingsley A.P.
        • Handunnetthi L.
        • et al.
        Prevalence of primary outcome changes in clinical trials registered on ClinicalTrials.gov: a cross-sectional study [v1; ref status: indexed, http://f1000r.es/34l].
        F1000Res. 2014; 3: 77
        • Pranić S.
        • Marušić A.
        Changes to registration elements and results in a cohort of Clinicaltrials.gov trials were not reflected in published articles.
        J Clin Epidemiol. 2016 Feb; 70: 26-37
        • Dwan K.
        • Altman D.G.
        • Cresswell L.
        • Blundell M.
        • Gamble C.L.
        • Williamson P.R.
        Comparison of protocols and registry entries to published reports for randomised controlled trials.
        Cochrane Database Syst Rev. 2011 Jan 19; 1: MR000031
        • Becker J.E.
        • Krumholz H.M.
        • Ben-Josef G.
        • Ross J.S.
        Reporting of results in ClinicalTrials.gov and high-impact journals.
        JAMA. 2014 Mar 12; 311: 1063-1065
        • National Institute of Health
        NIH's definition of a clinical trial.
      1. Government of Canada, Panel on research ethics. Tri-Council policy statement 2 - scope and approach. http://www.pre.ethics.gc.ca/eng/policy-politique/initiatives/tcps2-eptc2/chapter2-chapitre2/%20-%20toc02-1a. Last accessed April 24, 2019.

      2. ClinicalTrials.gov. https://clinicaltrials.gov] Last accessed April 24, 2019.

      3. CONSORT. Transparent reporting of trials. CONSORT 2010. http://www.consort-statement.org/consort-2010. Last accessed April 24, 2019.

        • Colombel J.F.
        • Panaccione R.
        • Bossuyt P.
        • et al.
        Effect of tight control management on Crohn's disease (CALM): a multicentre, randomised, controlled phase 3 trial.
        Lancet. 2018 Dec 23; 390: 2779-2789
        • ClinicalTrials.gov archive
        History of changes for study: NCT01235689.
        • Woodcock A.
        • Vestbo J.
        • Bakerly N.D.
        • et al.
        Effectiveness of fluticasone furoate plus vilanterol on asthma control in clinical practice: an open-label, parallel group, randomised controlled trial.
        Lancet. 2017 Nov 18; 390: 2247-2255
        • ClinicalTrials.gov archive
        History of changes for study: NCT01706198.
        • Bergeron A.
        • Chevret S.
        • Granata A.
        • et al.
        Effect of azithromycin on airflow decline-free survival after allogeneic hematopoietic stem cell transplant: the ALLOZITHRO randomized clinical trial.
        JAMA. 2017 Aug 8; 318: 557-566
        • Khanna A.
        • English S.W.
        • Wang X.S.
        • et al.
        Angiotensin II for the treatment of vasodilatory shock.
        N Engl J Med. 2017 Aug 3; 377: 419-430
        • Montori V.M.
        • Devereaux P.J.
        • Adhikari N.K.
        • et al.
        Randomized trials stopped early for benefit: a systematic review.
        JAMA. 2005 Nov 2; 294: 2203-2209
        • Woloshin S.
        • Schwartz L.M.
        • Bagley P.J.
        • Blunt H.B.
        • White B.
        Characteristics of interim publications of randomized clinical trials and comparison with final publications.
        JAMA. 2018 Jan 23; 319: 404-406
        • Connors J.M.
        • Jurczak W.
        • Straus D.J.
        • et al.
        Brentuximab Vedotin with chemotherapy for stage III or IV Hodgkin's Lymphoma.
        N Engl J Med. 2018 Jan 25; 378: 331-344
        • ClinicalTrials.gov archive
        History of changes for study: NCT01712490.
        • Eikelboom J.W.
        • Connolly S.J.
        • Bosch J.
        • et al.
        Rivaroxaban with or without aspirin in stable cardiovascular disease.
        N Engl J Med. 2017 Oct 5; 377: 1319-1330
        • ClinicalTrials.gov archive
        History of changes for study: NCT01776424.
        • Walsh M.
        • Srinathan S.K.
        • McAuley D.F.
        • et al.
        The statistical significance of randomized controlled trial results is frequently fragile: a case for a fragility index.
        J Clin Epidemiol. 2014 Jun; 67: 622-628
        • Lai Z.W.
        • Kelly R.
        • Winans T.
        • et al.
        Sirolimus in patients with clinically active systemic lupus erythematosus resistant to, or intolerant of, conventional medications: a single-arm, open-label, phase 1/2 trial.
        Lancet. 2018 Mar 24; 391: 1186-1196
        • Corren J.
        • Parnes J.R.
        • Wang L.
        • et al.
        Tezepelumab in adults with uncontrolled asthma.
        N Engl J Med. 2017 Sep 7; 377: 936-946
        • Tignanelli C.J.
        • Napolitano L.M.
        The fragility index in randomized clinical trials as a means of optimizing patient care.
        JAMA Surg. 2018 Nov 7; ([Epub ahead of print])https://doi.org/10.1001/jamasurg.2018.4318
        • Del Paggio J.C.
        • Tannock I.F.
        The fragility of phase 3 trials supporting FDA-approved anticancer medicines: a retrospective analysis.
        Lancet Oncol. 2019 Aug; 20: 1065-1069
        • Anderson D.R.
        • Dunbar M.
        • Murnaghan J.
        • et al.
        Aspirin or rivaroxaban for VTE prophylaxis after hip or knee arthroplasty.
        N Engl J Med. 2018 Feb 22; 378: 699-707
        • Venkatesh B.
        • Finfer S.
        • Cohen J.
        • et al.
        Adjunctive glucocorticoid therapy in patients with septic shock.
        N Engl J Med. 2018 Mar 1; 378: 797-808
        • Huttner A.
        • Kowalczyk A.
        • Turjeman A.
        • et al.
        Effect of 5-day nitrofurantoin vs single-dose Fosfomycin on clinical resolution of uncomplicated lower urinary tract infection in women: a randomized clinical trial.
        JAMA. 2018 May 1; 319: 1781-1789
        • Stott D.J.
        • Rodondi N.
        • Kearney P.M.
        • et al.
        Thyroid hormone therapy for older adults with subclinical hypothyroidism.
        N Engl J Med. 2017 Jun 29; 376: 2534-2544
        • Mathieu S.
        • Chan A.W.
        • Ravaud P.
        Use of trial register information during the peer review process.
        PLoS One. 2013 Apr 10; 8e59910
        • Matosin N.
        • Frank E.
        • Engel M.
        • Lum J.S.
        • Newell K.A.
        Negativity towards negative results: a discussion of the disconnect between scientific worth and scientific culture.
        Dis Model Mech. 2014 Feb; 7: 171-173
        • Sim I.
        • Chan A.W.
        • Gulmezoglu A.M.
        • Evans T.
        • Pang T.
        Clinical trial registration: transparency is the watchword.
        Lancet. 2006; 367: 1631-1633
        • Dal-Ré R.
        • Marušić A.
        Prevention of selective outcome reporting: let us start from the beginning.
        Eur J Clin Pharmacol. 2016; 72: 1283-1288
      4. FDA. Regulatory information. Establishment and Operation of Clinical Trial Data Monitoring Committees for Clinical Trial Sponsors. https://www.fda.gov/RegulatoryInformation/Guidances/ucm127069.htm#2. Last accessed April 24, 2019.

        • DeMets D.L.
        Data monitoring committees: history and their future.
        Trials. 2013; 14: I3
        • Department of Health and Human Services
        Final rule — clinical trials registration and results information submission.
        Fed Regist. 2016; 81: 64981-65157
        • Landman G.W.
        Escitalopram for the prevention of pegnterferon-α2a associated depression [Letter].
        Ann Intern Med. 2013; 158: 139-140
        • Prasad V.
        Treatment of moderate to severe hidiadenitis suppurativa [Letter].
        Ann Intern Med. 2013; 159: 72
        • Obi Y.
        • Hamano T.
        The change history of the INDEPENDENT Study in the ClinicalTrials.gov database.
        Am J Kidney Dis. 2014 Jan; 63: 164
        • Shepshelovich D.
        • Yelin D.
        • Gafter-Gvili A.
        • Goldman S.
        • Avni T.
        • Yahav D.
        Comparison of reporting phase III randomized controlled trials of antibiotic treatment for common bacterial infections in ClinicalTrials.gov and matched publications.
        Clin Microbiol Infect. 2018 Nov; 241211
        • Askie L.
        Trial registration records, updates, and protocols.
        Lancet. 2016; 388: 341-342

      Linked Article

      • Clinical trial publications: A sufficient basis for healthcare decisions?
        European Journal of Internal MedicineVol. 71
        • Preview
          Clinical studies remain the single most important information source of the scientific basis on which healthcare decisions are taken. This applies to population-level decisions taken by drug regulators or healthcare payers as well as individual-level decisions taken by patients and their physicians. With respect to drug treatments, adequately designed, executed, and reported randomised controlled trials (RCTs) have often been described as the “gold standard” for decision making. High expectations and ethical responsibilities are riding on publications of RCTs in peer reviewed journals.
        • Full-Text
        • PDF