Advertisement

Peer review in medical journals: Beyond quality of reports towards transparency and public scrutiny of the process

Published:April 26, 2016DOI:https://doi.org/10.1016/j.ejim.2016.04.014

      Highlights

      • The peer review process determines the selection of studies that will inform health care decisions.
      • The single-blind model (reviewers unknown to authors) is adopted by most biomedical journals.
      • Blinding also reviewers (double-blind model) is tricky and does not improve reports' quality.
      • Open peer review ensures transparency about study reports that may influence clinical practice.
      • Also the editorial process should be made public, avoiding an excessive focus on reviewers' role.

      Abstract

      Published medical research influences health care providers and policy makers, guides patient management, and is based on the peer review process. Peer review should prevent publication of unreliable data and improve study reporting, but there is little evidence that these aims are fully achieved. In the blinded systems, authors and readers do not know the reviewers' identity. Moreover, the reviewers' reports are not made available to readers. Anonymous peer review poses an ethical imbalance toward authors, who are judged by masked referees, and to the medical community and society at large, in case patients suffer the consequences of acceptance of flawed manuscripts or erroneous rejection of important findings. Some general medical journals have adopted an open process, require reviewers to sign their reports, and links online pre-publication histories to accepted articles. This system increases editors' and reviewers' accountability and allows public scrutiny, consenting readers understand on which basis were decisions taken and by whom. Moreover, this gives credit to reviewers for their apparently thankless job, as online availability of signed and scored reports may contribute to researchers' academic curricula. However, the transition from the blind to the open system could pose problems to journals. Reviewers may be more difficult to find, and publishers or medical societies could resist changes that may affect editorial costs and journals' revenues. Nonetheless, also considering the risk of competing interests in the medical field, general and major specialty journals could consider testing the effects of open review on manuscripts regarding studies that may influence clinical practice.

      Abbreviations:

      COI (Conflicts of interest), DOI (Digital object identifiers), ORCID (Open researcher and contributor ID)

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to European Journal of Internal Medicine
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • AM Vintzileos
        • CV Ananth
        • AO Odibo
        • SP Chauhan
        • JC Smulian
        • Oyelese Y.
        The relationship between a reviewer's recommendation and editorial decision of manuscripts submitted for publication in obstetrics.
        Am J Obstet Gynecol. 2014; 211: 703.e1-703.e5
        • Manchikanti L.
        • Kaye A.D.
        • Boswell M.V.
        • Hirsch J.A.
        Medical journal peer review: process and bias.
        Pain Physician. 2015; 18: E1-E4
        • Walker R.
        • da Silva P Rocha
        Emerging trends in peer review – a survey.
        Front Neurosci. 2015; 9: 169
        • Resnik D.B.
        • Elmore S.A.
        Ensuring the quality, fairness, and integrity of journal peer review: a possible role of editors.
        Sci Eng Ethics. 2016; 22: 169-188
        • Steinbrook R.
        The peer review congresses: improving peer review and biomedical publication.
        JAMA. 2013; 310: 1799-1800
        • Smith R.
        Peer review: a flawed process at the heart of science and journals.
        J R Soc Med. 2006; 99: 178-182
        • Moylan E.C.
        • Harold S.
        • O'Neill C.
        • Kowalczuk M.K.
        Open, single-blind, double-blind: which peer review process do you prefer?.
        BMC Pharmacol Toxicol. 2014; 15: 55
        • Patel J.
        Why training and specialization is needed for peer review: a case study of peer review for randomized controlled trials.
        BMC Med. 2014; 12: 128
        • Groves T.
        Is open peer review the fairest system?.
        Yes BMJ. 2010; 341: c6424
        • van Rooyen S.
        • Godlee F.
        • Evans S.
        • Black N.
        • Smith R.
        Effect of open peer review on quality of reviews and on reviewers' recommendations: a randomised trial.
        BMJ. 1999; 318: 23-
        • van Rooyen S.
        • Delamothe T.
        • Evans S.J.
        Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial.
        BMJ. 2010; 341: c5729
        • Albanese M.
        Three blind mice–might make good reviewers.
        Med Educ. 2006; 40: 828-830
        • Khan K.
        Is open peer review the fairest system?.
        No BMJ. 2010; 341: c6425
        • Morrison J.
        The case for open peer review.
        Med Educ. 2006; 40: 830-831
        • Schroter S.
        • Tite L.
        • Hutchings A.
        • Black N.
        Differences in review quality and recommendations for publication between peer reviewers suggested by authors or by editors.
        JAMA. 2006; 295: 314-317
        • Pitkin R.M.
        Blinded manuscript review: an idea whose time has come?.
        Obstet Gynecol. 1995; 85: 781-782
        • Godlee F.
        • Gale C.R.
        • Martyn C.N.
        Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: a randomized controlled trial.
        JAMA. 1998; 280: 237-240
      1. Nature. 2015; 518: 274
        • DeCoursey T.E.
        Publishing: double-blind peer review a double risk.
        Nature. 2015; 520: 623
        • Cho M.K.
        • Justice A.C.
        • Winker M.A.
        • Berlin J.A.
        • Waeckerle J.F.
        • Callaham M.L.
        • et al.
        Masking author identity in peer review: what factors influence masking success? PEER investigators.
        JAMA. 1998; 280: 243-245
        • Regehr G.
        • Bordage G.
        To blind or not to blind? What authors and reviewers prefers.
        Med Educ. 2006; 40: 832-839
        • Justice A.C.
        • Cho M.K.
        • Winker M.A.
        • Berlin J.A.
        • Rennie D.
        Does masking author identity improve peer review quality? A randomized controlled trial. PEER Investigators.
        JAMA. 1998; 280: 240-242
        • Pitkin R.M.
        Masked peer review revisited.
        Obstet Gynecol. 1998; 91: 780
        • Chung K.C.
        • Shauver M.J.
        • Malay S.
        • Zhong L.
        • Weinstein A.
        • Rohrich R.J.
        Is double-blinded peer review necessary? The effect of blinding on review quality.
        Plast Reconstr Surg. 2015; 136: 1369-1377
        • Squazzoni F.
        • Gandelli C.
        Opening the black-box of peer review: an agent-based model of scientist behaviour.
        J Artif Soc Soc Simul. 2013; 16: 3
        • Cohen J.
        • Grudzinskas G.
        • Johnson M.H.
        Possible conflicts of interest in medical publishing.
        Reprod Biomed Online. 2013; 26: 409-410
        • van Rooyen S.
        • Godlee F.
        • Evans S.
        • Smith R.
        • Black N.
        Effect of blinding and unmasking on the quality of peer review: a randomized trial.
        JAMA. 1998; 280: 234-237
        • Walsh E.
        • Rooney M.
        • Appleby L.
        • Wilkinson G.
        Open peer review: a randomised controlled trial.
        Br J Psychiatry. 2000; 176: 47-51
        • Kowalczuk M.K.
        • Dudbridge F.
        • Nanda S.
        • Harriman S.L.
        • Patel J.
        • Moylan E.C.
        Retrospective analysis of the quality of reports by author-suggested and non-author-suggested reviewers in journals operating on open or single-blind peer review models.
        BMJ Open. 2015; 5e008707
        • Godlee F.
        Making reviewers visible: openness, accountability, and credit.
        JAMA. 2002; 287: 2762-2765
        • Groves T.
        • Loder E.
        Prepublication histories and open peer review at the BMJ.
        BMJ. 2014; 349: g5394
      2. Goetz A. Reexamining reviewer anonymity – more costs than benefits. Open Science Collaboration. Available: http://osc.centerforopenscience.org/2014/10/22/reexamining-reviewer-anonymity/ [accessed 23/03/2016].

        • Tite L.
        • Schroter S.
        Why do peer reviewers decline to review? A survey.
        J Epidemiol Community Health. 2007; 61: 9-12
        • van Rooyen S.
        • Black N.
        • Godlee F.
        Development of the review quality instrument (RQI) for assessing peer reviews of manuscripts.
        J Clin Epidemiol. 1999; 52: 625-629
        • Landkroon A.P.
        • Euser A.M.
        • Veeken H.
        • Hart W.
        • Overbeke A.J.
        Quality assessment of reviewers' reports using a simple instrument.
        Obstet Gynecol. 2006; 108: 979-985
      3. Publons: track and verify your peer review. Available: https://publons.com [accessed 23/03/2016]

        • Van Noorden R.
        The scientists who get credit for peer review.
        2014https://doi.org/10.1038/nature.2014.16102 ([Available: http://www.nature.com/news/the-scientists-who-get-credit-for-peer-review-1.16102 [accessed 23/03/2016]])
        • Chimes C.
        News roundup: publons data in altmetric details pages.
        2013 ([Available: http://www.altmetric.com/blog/publons/ [accessed 23/03/2016]])
      4. Nature. 2014; 514: 274
        • Gasparyan A.Y.
        • Gerasimov A.N.
        • Voronov A.A.
        • Kitas G.D.
        Rewarding peer reviewers: maintaining the integrity of science communication.
        J Korean Med Sci. 2015; 30: 360-364
      5. Peer review evaluation (PRE). Available: www.pre-val.org/[accessed 23/03/2016]

        • Mehmani B.
        • van Rossum J.
        Elsevier trials publishing peer review reports as articles.
        2015 (Available:) ([accessed 23/03/2016])
      6. ScienceDirect.com. Science, health and medical journals, full text articles and books. Available: http://www.sciencedirect.com [accessed 23/03/2016]

      7. Engineering Fracture Mechnisms. Peer review Report 2015;133, Supplement 1:1–308. Available: http://www.sciencedirect.com/science/journal/00137944/133/supp/S1 [accessed 23/03/2016]

        • Smith R.
        Conflicts of interest: how money clouds objectivity.
        J R Soc Med. 2006; 99: 292-297
        • Handel A.E.
        • Patel S.V.
        • Pakpoor J.
        • Ebers G.C.
        • Goldacre B.
        • Ramagopalan S.V.
        High reprint orders in medical journals and pharmaceutical industry funding: case–control study.
        BMJ. 2012; 344: e4212
        • Smith R.
        Time to open up finances of medical journals.
        BMJ. 2012; 345: e4968
        • McCartney M.
        Margaret McCartney: medical journals and their parasitical profit.
        BMJ. 2015; 350: h2832
        • Smith R.
        The highly profitable but unethical business of publishing medical research.
        J R Soc Med. 2006; 99: 452-456
        • Smith R.
        Medical journals are an extension of the marketing arm of pharmaceutical companies.
        PLoS Med. 2005; 2: e138
        • Schroter S.
        • Black N.
        • Evans S.
        • Carpenter J.
        • Godlee F.
        • Smith R.
        Effects of training on quality of peer review: randomised controlled trial.
        BMJ. 2004; 328: 673
        • Jefferson T.
        • Alderson P.
        • Wager E.
        • Davidoff F.
        Effects of editorial peer review: a systematic review.
        JAMA. 2002; 287: 2784-2786
        • Jefferson T.
        • Wager E.
        • Davidoff F.
        Measuring the quality of editorial peer review.
        JAMA. 2002; 287: 2786-2790
        • Jefferson T.
        • Rudin M.
        • Brodney Folse S.
        • Davidoff F.
        Editorial peer review for improving the quality of reports of biomedical studies.
        Cochrane Database Syst Rev. Apr 2007; 18: MR000016
        • Schroter S.
        • Black N.
        • Evans S.
        • Godlee F.
        • Osorio L.
        • Smith R.
        What errors do peer reviewers detect, and does training improve their ability to detect them?.
        J R Soc Med. 2008; 101: 507-514
        • Haug C.J.
        Peer-review fraud—hacking the scientific publication process.
        N Engl J Med. 2015; 373: 2393-2395
        • Ferguson C.
        • Marcus A.
        • Oransky I.
        Publishing: the peer-review scam.
        Nature. 2014; 515: 480-482
        • Wager E.
        • Kleinert S.
        Why do we need international standards on responsible research publication for authors and editors?.
        J Glob Health. 2013; 3: 020301
        • Newton D.P.
        Quality and peer review of research: an adjudicating role for editors.
        Account Res. 2010; 17: 130-145
        • Lundh A.
        • Barbateskovic M.
        • Hróbjartsson A.
        • Gøtzsche P.C.
        Conflicts of interest at medical journals: the influence of industry-supported randomised trials on journal impact factors and revenue—cohort study.
        PLoS Med. 2010; 7: e1000354
        • Gleicher N.
        Avoiding currently unavoidable conflicts of interest in medical publishing by transparent peer review.
        Reprod Biomed Online. 2013; 26: 411-415
        • Rasmussen K.
        • Jørgensen K.J.
        • Gøtzsche P.C.
        Citations of scientific results and conflicts of interest: the case of mammography screening.
        Evid Based Med. 2013; 18: 83-89
        • Jarvies D.
        • Coombes R.
        • Stahl-Timmins W.
        Open payments goes live with pharma to doctor fee data: first analysis.
        BMJ. 2014; 349: g6003
        • Hurd W.W.
        Conflicts of interest and medical publishing.
        Obstet Gynecol. 2013; 122: 511-512