Advertisement

Waste in COVID-19 clinical trials conducted in western Europe

  • Author Footnotes
    1 R Dal-Ré: ORCID: 0000–0002–0980–2486
    Rafael Dal-Ré
    Correspondence
    Corresponding author.
    Footnotes
    1 R Dal-Ré: ORCID: 0000–0002–0980–2486
    Affiliations
    Epidemiology Unit, Health Research Institute-Fundación Jiménez Díaz University Hospital, Universidad Autónoma de Madrid, Avda Reyes Católicos 2, E-28040 Madrid, Spain
    Search for articles by this author
  • Author Footnotes
    2 I Mahillo-Fernández: ORCID: 0000–0003–0558–6541
    Ignacio Mahillo-Fernández
    Footnotes
    2 I Mahillo-Fernández: ORCID: 0000–0003–0558–6541
    Affiliations
    Epidemiology Unit, Health Research Institute-Fundación Jiménez Díaz University Hospital, Universidad Autónoma de Madrid, Avda Reyes Católicos 2, E-28040 Madrid, Spain
    Search for articles by this author
  • Author Footnotes
    1 R Dal-Ré: ORCID: 0000–0002–0980–2486
    2 I Mahillo-Fernández: ORCID: 0000–0003–0558–6541

      Keywords

      Amid huge efforts to address the therapeutic uncertainties clinicians are facing with COVID-19 patients’ management, many clinical trials are badly designed [
      • Glasziou P.P.
      • Sanders S.
      • Hoffmann T
      Waste in covid-19 research.
      ]. Trials with no control group and with planned sample sizes of ≤100 participants are examples of poor research [
      • Glasziou P.P.
      • Sanders S.
      • Hoffmann T
      Waste in covid-19 research.
      ]. However, the need to conduct well designed trials that could provide robust findings is a must, from both the scientific and ethical perspectives [
      • London A.J.
      • Kimmelman J.
      Against pandemic research exceptionalism.
      ]. The quality of a trial relates to the design, conduct, analysis and reporting. The design is the only aspect that can be appraised in ongoing trials from the information reported in trial registries. Although the design can only provide a partial view of the trial's quality, it is relevant to assess the main design characteristics since ill-design trials will provide very limited useful information.
      A cross-sectional analysis was carried out based on a series of searches conducted on May 20–21, 2020 looking for non-industry-sponsored, ongoing trials, aiming to assess medicines or convalescent plasma for the treatment of COVID-19 patients, in the 5 largest European countries and 2 regions (Benelux, Scandinavia). The searches were conducted on five registries (ClinicalTrials.gov, DKRS, EU-CTR, ISRCTN and NTR) with the terms ‘coronavirus’, ‘covid’ and ‘covid-19′. So, three searches were conducted per registry and for each country. ‘Country’: Belgium, Denmark, Finland, France, Germany, Italy, Luxembourg, Netherlands, Norway, Spain, Sweden, and the UK. On the searches on ClinicalTrials.gov, the following descriptors were also used: Study type: ‘Interventional studies (clinical trials)’; Status: recruitment: ‘Not yet recruiting’, ‘recruiting’ and ‘active: not recruiting’; Funder type: ‘NIH’, ‘Other US Federal Agency’ and ‘All others (individuals, universities, organizations)’. Trials on special populations (e.g. cancer, diabetes , pregnancy, healthy volunteers, healthcare workers), or with special objectives (e.g. pharmacokinetics) or assessing prophylactic medicines (both pre- and post-exposure) or both prophylaxis and treatment, or assessing vaccines, or dietary supplements (eg, vitamins), or herbal preparations (eg Aesculus hippocastanum) were excluded. All phase 1 or phase 1/2 trials were excluded. Ozone trials were included.
      The following information was extracted from each trial: single-arm or randomized controlled trial (RCT), blinding, and planned sample size; whether the trial was multicenter, had a control (standard of care) arm and a Data Monitoring Committee. All these features are associated, in principle, with robust designs. Randomization and blinding prevent selection, performance and detection bias. Large sample sizes help to eliminate known and unknown confounders. Single-center trials tend to provide larger intervention effects than multicenter RCTs. Being COVID-19 a life-threatening disease, a Data Monitoring Commitee should be present in every multicenter RCT [

      European Medicines Agency. Guideline on data monitoring committees. EMEA/CHPM/EWP/5872/03 Corr. 27 July2005https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-data-monitoring-committees_en.pdf Accessed: Accessed: 16 June 2020.

      ]. Since there are no approved drugs, all trials should have a standard of care group.
      Fisher's exact test was used for comparisons. A p<0.05 was considered significant (two-sided test).
      There were 159 different trials, 141 (89%) RCTs and 18 single-arm trials. The findings (Table 1) showed that Italy was conducting 44% of all single-arm trials run in western Europe and accounted for 38% of all trials conducted in Italy. France and Spain carried out between 3 and 5 times the number of RCTs run in the other countries/regions. The majority of RCTs conducted in Germany (63%) were masked, but only some 25% of those conducted in Benelux, France, Italy, Spain and the UK. Eighty-six percent of RCTs run in Scandinavia were multicenter, but only 54% of those carried out in Spain. No standard of care arm was only present in RCTs conducted in France, Italy and Spain (p = 0.015). Data Monitoring Committee was differently present across multicenter RCTs in all countries/regions (p = 0.019). There were few large trials (Table 2). Of special interest were SOLIDARITY, the World Health Organization-sponsored RCT, that was being conducted in five of the countries included in this study, and RECOVERY, the largest trial conducted in only one country (the UK) aiming to recruit 12,000 participants. Forty-two percent of all trials single-arm trials, RCTs recruiting ≤50 participants/arm and those with no standard of care arm were poorly designed.
      Table 1Ongoing trials on the treatment with medicinal products in COVID-19 patients conducted in 12 western European countries and registered between 19 February and 19 May 2020.
      Country/RegionRandomised controlled trialsSingle-arm trial
      NaBlinding (%)MCT N (%)No SoC arm (%)DMC Nb (%)n/armN
      YesUnknowndNo≤5051–400≥401
      Benelux133 (23)9 (69)05 (56)2 (22)2 (22)2 (15)10 (77)1 (8)0
      France4111c (27)31 (76)5 (12)9 (29)11 (35)11 (35)12 (29)26 (64)3 (7)2
      Germany85 (63)6 (75)04 (67)2 (23)01 (12)5 (63)2 (25)1
      Italy133c (23)9 (69)3 (23)2 (22)7 (78)02 (15)9 (69)2 (15)8
      Scandinavia147 (50)12 (86)05 (42)2 (17)5 (42)3 (21)10 (72)1 (7)3
      Spain4110c (24)22 (54)12 (29)9 (41)2 (9)11 (50)13 (32)25 (61)3 (7)2
      UK123 (25)8 (75)03 (38)1 (12)4 (50)3 (25)7 (58)2 (17)2
      P-value0.2560.2230.0150.0190.882
      Benelux: Belgium, Netherlands, Luxembourg; DMC: Data monitoring committee; MCT: Multicenter trial; N: number of trials; n: number of participants; Scandinavia: Denmark, Finland, Norway, Sweden; SoC: standard of care.
      (a) The following trials were conducted in more than one country: ACTT (Germany, Denmark, Spain, UK), OsCOVID (France, Italy), DISCOVERY (France, Luxembourg), SOLIDARITY (Finland, Germany, Italy, Norway, Spain); (b) Only multicenter trials were considered; (c) Plus one open-label randomized controlled trial with assessor blinded to treatment arms; (d) This information was only provided on EU-CTR.
      Table 2Main features of large (≥ 401 participants/arm) ongoing randomized controlled trials on the treatment with medicinal products in COVID-19 patients conducted in 12 western European countries and registered between 19 February and 19 May 2020.
      Trial nameIDCountry/ RegionAssessed medicinesDesignN (total)Type of patients
      COLCHI-COVID2020–001603–16SpainColchicine, vs SoC2-arm; parallel; open-label1024Outpatients
      COLCORONAa2020–001689–12 NCT04322682SpainColchicine, vs SoC + placebo2-arm; parallel; double-blind6000Outpatients
      COMIHYNCT04340544GermanyHydroxychloroquine, vs SoC + placebo2-arm; parallel; double-blind2700Outpatients
      CORIMUNO19-COAGNCT04344756FranceTinzaparin, vs SoC2-arm; parallel; open-label808Hospitalized
      COVID-STEROID2020–001395–15 NCT04348305Scandinavia (Denmark)Hydrocortisone, vs SoC + placebo2-arm; parallel; double-blind1000Hospitalized
      DISCOVERYa2020–000936–23 NCT04315948Benelux (Luxembourg), FranceRemdesivir, interferon beta 1A, lopinavir/ritonavir, hydroxychloroquine, vs SoC5-arm; adaptive; parallel; open-label3100Hospitalized
      HYCOVID2020–001271–33 NCT04325893FranceHydroxychloroquine, vs SoC + placebo2-arm; parallel; double-blind1300Hospitalized
      RECOVERY2020–001113–21 ISRCTN50189673UKHydroxychloroquine, azithromycin, tocilizumab, lopinavir/ritonavir, dexamethasone, vs SoC5-arm; adaptive; parallel; open-label12000Hospitalized
      SOLIDARITYa,b2020–001366–11 ISRCTN83971151Germany, Italy, SpainRemdesivir, chloroquine or hydroxychloroquine, lopinavir/ritonavir, interferon beta-1A, vs SoC5-arm; adaptive; parallel; open-label100000Hospitalized
      TACTIC-ENCT04393246UKDapagliflozin + ambrisentan, EDP1815, vs SoC3-arm; parallel; open-label1407Hospitalized
      X-COVID19NCT04366960ItalyEnoxaparin, 2 different dosages; no SoC arm2-arm; parallel; open-label2712Hospitalized
      Benelux: Belgium, Netherlands, Luxembourg; N: planned sample size; Scandinavia: Denmark, Finland, Norway, Sweden; SoC: standard of care.
      (a) Trial conducted in other countries in addition to the ones considered in this analysis; (b) This trial is also run in Finland and Norway but with different number of arms (n = 3), and thus with different ID.
      This study shows that low-quality clinical trials were being conducted in all analyzed western European countries/regions, which are among the territories of the world with the best clinical research. Mediterranean countries (France, Italy and Spain) had higher number of single-arm and significantly more RCTs without standard of care arm than that of the rest of countries/regions, which resulted in a seemingly trend to higher percentage of low-quality trials than in central/northern Europe. Single-arm trials accounted for a similar percentage in this analysis (11%) and in a study that included 201 trials registered in any continent (14%) [
      • Mehta H.B.
      • Ehrhardt S.
      • Moore T.J.
      • Segal J.B.
      • Alexander G.C
      Characteristics of registered clinical trials assessing treatments for COVID-19: a cross-sectional analysis.
      ]. Limited useful information should be expected from single-arm trials, RCTs recruiting ≤50 participants/arm and those with no standard of care arm. Some of these, however, could provide hypothesis-generating findings rather than hypothesis-supporting results , that should be confirm in future adequate RCTs.
      Since it should be expected that the absolute reduction on mortality should be limited [
      • Bauchner H.
      • Fontanerosa P.B.
      Randomized clinical trials and COVID-19. Managing expectations.
      ], many trials recruiting even hundreds of participants will end up with neutral/negative results. Yet, some trials could show positive results, but this will likely happen in non-critical (but important) endpoints as occurred in an RCT with >500 participants/arm that found a shorter time to recovery with remdesivir compared with placebo, although non-significant difference was observed on mortality [
      • Beigel J.H.
      • Tomashek K.M.
      • Dodd L.E.
      • Mehta A.K.
      • Zingman B.S.
      • Kalil A.C.
      • et al.
      Remdesivir for the treatment of Covid-19 — preliminary report.
      ]. When reaching larger number of participants (>1500/arm), the benefit on mortality could be appropriately assessed as has been shown in the RECOVERY trial: dexamethasone has proven to be beneficial [

      University of Oxford news release. RECOVERY. Low-cost dexamethasone reduces death by up to one third in hospitalised patients with severe respiratory complications of COVID-19. 16 June 2020. https://www.recoverytrial.net/files/recovery_dexamethasone_statement_160620_v2final.pdf Accessed: 16 June 2020.

      ] whereas hydroxychloroquine did not [

      University of Oxford news release. RECOVERY. No clinical benefit from use of hydroxychloroquine in hospitalised patients with COVID-19. 5 June 2020. https://www.recoverytrial.net/files/hcq-recovery-statement-050620-final-002.pdf Accessed: 16 June 2020.

      ].
      The main limitation of this study is that all the information has been captured from trial registries that are not always updated. So, for example, after recruiting 1059 participants the ACTT trial was early terminated [
      • Beigel J.H.
      • Tomashek K.M.
      • Dodd L.E.
      • Mehta A.K.
      • Zingman B.S.
      • Kalil A.C.
      • et al.
      Remdesivir for the treatment of Covid-19 — preliminary report.
      ], soon before the searches on trial registries of this analysis were conducted, but it appeared on the registries as ‘recruiting’ (ClinicalTrials.gov) and ‘ongoing’ (EU-CTR), with a planned sample size of 800 participants. Also, the fact that EU-CTR is the only registry reporting on the presence of the RCT's Data Monitoring Committee limits its usefulness as a factor of robust design, since it was not available from trials that were only registered in other registries. However, it should be highlighted that it is rather surprising why some trials carried out in any of the countries/regions included in this study seemed not to be registered on the EudraCT database (from which the EU-CTR obtains all the information), something that should be expected from all trials conducted with medicines in Europe. Since convalescent plasma is not a medicinal product and ozone could also be considered as such, trials assessing only these interventions could reasonably not be on EU-CTR; this, however, cannot be applied to other trials conducted with medicines that were being assessed on the treatment of COVID-19 patients (see Supplementary material).
      So as not to waste the generosity from trial participants and the effort from investigators of all these trials, de-identified individual participant clinical trial data should be forwarded to teams that are conducting living systematic reviews with network meta-analysis [
      • Juul S.
      • Nielsen N.
      • Bentzer P.
      • Veroniki A.A.
      • Thabane L.
      • Linder A.
      • et al.
      Interventions for treatment of COVID-19: a protocol for a living systematic review with network meta-analysis including individual patient data (The LIVING Project). PROSPERO CRD42020178787.
      ,

      Boutron I., Chaimani A., Devane D., Meerpohl J.J., Tovey D., Hróbjartsson A. et al. Interventions for preventing and treating COVID-19: protocol for a living mapping of research and a living systematic review. https://zenodo.org/record/3744600#.XrUINmgzYtw Accessed: 16 June 2020.

      ], that could report their results in a short period of time. This will be of invaluable benefit to clinicians and future patients.

      Funding

      This work required no funding.

      Declaration of Competing Interests

      None declared.

      Data availability

      All data supporting the results is available on the article and the supplementary material

      Appendix. Supplementary materials

      References

        • Glasziou P.P.
        • Sanders S.
        • Hoffmann T
        Waste in covid-19 research.
        BMJ. 2020; 369: m1847
        • London A.J.
        • Kimmelman J.
        Against pandemic research exceptionalism.
        Science. 2020; 368: 476-477
      1. European Medicines Agency. Guideline on data monitoring committees. EMEA/CHPM/EWP/5872/03 Corr. 27 July2005https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-data-monitoring-committees_en.pdf Accessed: Accessed: 16 June 2020.

        • Mehta H.B.
        • Ehrhardt S.
        • Moore T.J.
        • Segal J.B.
        • Alexander G.C
        Characteristics of registered clinical trials assessing treatments for COVID-19: a cross-sectional analysis.
        BMJ Open. 2020; 10e039978
        • Bauchner H.
        • Fontanerosa P.B.
        Randomized clinical trials and COVID-19. Managing expectations.
        JAMA. 2020; 323: 2262-2263
        • Beigel J.H.
        • Tomashek K.M.
        • Dodd L.E.
        • Mehta A.K.
        • Zingman B.S.
        • Kalil A.C.
        • et al.
        Remdesivir for the treatment of Covid-19 — preliminary report.
        N Engl J Med. 2020 May 22; (Online ahead of print)https://doi.org/10.1056/NEJMoa2007764
      2. University of Oxford news release. RECOVERY. Low-cost dexamethasone reduces death by up to one third in hospitalised patients with severe respiratory complications of COVID-19. 16 June 2020. https://www.recoverytrial.net/files/recovery_dexamethasone_statement_160620_v2final.pdf Accessed: 16 June 2020.

      3. University of Oxford news release. RECOVERY. No clinical benefit from use of hydroxychloroquine in hospitalised patients with COVID-19. 5 June 2020. https://www.recoverytrial.net/files/hcq-recovery-statement-050620-final-002.pdf Accessed: 16 June 2020.

        • Juul S.
        • Nielsen N.
        • Bentzer P.
        • Veroniki A.A.
        • Thabane L.
        • Linder A.
        • et al.
        Interventions for treatment of COVID-19: a protocol for a living systematic review with network meta-analysis including individual patient data (The LIVING Project). PROSPERO CRD42020178787.
        Syst Rev. 2020; 9: 108
      4. Boutron I., Chaimani A., Devane D., Meerpohl J.J., Tovey D., Hróbjartsson A. et al. Interventions for preventing and treating COVID-19: protocol for a living mapping of research and a living systematic review. https://zenodo.org/record/3744600#.XrUINmgzYtw Accessed: 16 June 2020.