Advertisement

A systematic review of the discrimination and absolute mortality predicted by the National Early Warning Scores according to different cut-off values and prediction windows

Open AccessPublished:December 31, 2021DOI:https://doi.org/10.1016/j.ejim.2021.12.024

      Highlights

      • NEWS reliably identifies patients who are unlikely to die within 24 h.
      • Prediction of mortality after 24 h by NEWS is unreliable.
      • Many hospitalized patients presenting with a low NEWS die within 30 days.
      • NEWS reliably identifies patients who need immediate attention.

      Abstract

      Background

      Although early warning scores were intended to simply identify patients in need of life-saving interventions, prediction has become their commonest metric. This review examined variation in the ability of the National Early Warning Scores (NEWS) in adult patients to predict absolute mortality at different times and cut-offs values.

      Method

      Following PRISMA guidelines, all studies reporting NEWS and NEWS2 providing enough information to fulfil the review's aims were included.

      Results

      From 121 papers identified, the average area under the Receiver Operating Characteristic curve (AUC) for mortality declined from 0.90 at 24-hours to 0.76 at 30-days. Studies with a low overall mortality had a higher AUC for 24-hour mortality, as did general ward patients compared to patients seen earlier in their treatment. 24-hour mortality increased from 1.8% for a NEWS ≥3 to 7.8% for NEWS ≥7. Although 24-hour mortality for NEWS <3 was only 0.07% these deaths accounted for 9% of all deaths within 24-hours; for NEWS <7 24-hour mortality was 0.23%, which accounted for 44% of all 24-hour deaths. Within 30-days of a NEWS recording 22% of all deaths occurred in patients with a NEWS <3, 52% in patients with a NEWS <5, and 75% in patient with a NEWS <7.

      Conclusion

      NEWS reliably identifies patients most and least likely to die within 24-hours, which is what it was designed to do. However, many patients identified to have a low risk of imminent death die within 30-days. NEWS mortality predictions beyond 24-hours are unreliable.

      Keywords

      1. Introduction

      Early warning scores (EWS) were originally proposed by Morgan et al. to help identify patients who need immediate life-saving interventions, but not to predict their outcomes [
      • Morgan R.J.
      • Wright M.M.
      In defence of early warning scores.
      ]. Ironically outcome prediction, rather than outcome improvement, has become a common metric of their performance [
      • Gerry S.
      • Bonnici T.
      • Birks J.
      • et al.
      Early warning scores for detecting deterioration in adult hospital patients: systematic review and critical appraisal of methodology.
      ,
      • Wright M.M.
      • Morgan R.J.M
      In defence of early warning scores.
      ]. EWS performance is most frequently reported as its ability to discriminate those patients who will develop an outcome from those who will not. Discrimination can be quantified by the area under the Receiver Operating Characteristic (ROC) curve (AUC). This test is most useful in the early evaluation of a score and may be dependant on patient characteristics and disease spectrum [
      • Mandrekar J.N.
      Receiver operating characteristic curve in diagnostic test assessment.
      ]. From a clinical perspective, especially if the discriminatory value is already high, small variations in the AUC between different EWSs may be unimportant. Moreover, odds ratios and other statistical manipulations may deflect attention from the variation in absolute mortality rates reported by different studies. What may matter most clinically is the score's performance at different time prediction windows, the absolute risk of death predicted, and what other factors might influence the accuracy of this prediction.
      The original National Early Warning Score (NEWS) developed by the Royal College of Physicians in the United Kingdom [
      Royal College of Physicians
      National Early Warning Score (NEWS): standardising the assessment of acute illness severity in the NHS.
      ] is the most well validated EWS in clinical use [
      • Gerry S.
      • Bonnici T.
      • Birks J.
      • et al.
      Early warning scores for detecting deterioration in adult hospital patients: systematic review and critical appraisal of methodology.
      ,
      • Dean J.D.
      • Williams B.
      • Inada-Kim M.
      Rapid Response: early warning scores supplement clinical judgement.
      ]. Unlike Morgan's original EWS it was not based on expert opinion, but on the analysis of a huge database of vital signs collected over several years at Portsmouth Hospitals NHS Trust. From this data the Vitalpac™ Early Warning Score (ViEWS) was derived, and subsequently slightly modified for operational reasons so that it could be promoted as NEWS throughout the NHS. ViEWS was originally designed to identify sick patients likely to die from any cause within 24-hours and not to include age as a predictor variable [
      • Prytherch D.R.
      • Smith G.B.
      • Schmidt P.E.
      • Featherstone P.I.
      ViEWS—Towards a national early warning score for detecting adult inpatient deterioration.
      ]. In 2017 NEWS was modified to NEWS2, which included a redefinition of altered mental status and the points awarded for oxygen saturation [
      Royal College of Physicians
      National Early Warning Score (NEWS) 2. Standardising the assessment of acute illness severity in the NHS.
      ]. These changes were not based on data, but on expert opinion. Although NEWS and NEWS2 are similar and intimately connected, it cannot be assumed that they are interchangeable, or that one is superior to the other [
      • Pimentel M.A.F.
      • Redfern O.C.
      • Gerry S.
      • et al.
      A comparison of the ability of the National Early Warning Score and the National Early Warning Score 2 to identify patients at risk of in-hospital mortality: a multi-centre database study.
      ,
      • O’Driscoll R.
      • Bakerly N.
      • Murphy P.
      • Turkington P.
      NEWS2 needs to be tested in prospective trials involving patients with confirmed hypercapnia.
      ,
      • Hodgson L.E.
      • Congleton J.
      • Venn R.
      • Forni L.G.
      • Roderick P.J.
      NEWS 2 – too little evidence to implement?.
      ].
      The predictive discrimination of a score is likely dependant on the outcomes measured, the length of the prediction window used, and how many effective interventions the patients studied received. Studies that have compared different scores have often used prediction windows, patient populations and outcomes that none of the scores tested were originally intended for [
      • Gerry S.
      • Bonnici T.
      • Birks J.
      • et al.
      Early warning scores for detecting deterioration in adult hospital patients: systematic review and critical appraisal of methodology.
      ,
      • Fang A.H.S.
      • Lim W.T.
      • Balakrishnan T.
      Early warning score validation methodologies and performance metrics: a systematic review.
      ]. The risk of death within 24-hours, arguably the most useful trigger for immediate escalation of care, occurs rarely in most patients. Therefore, ‘composite outcomes’ that include interventions such ICU admission are frequently used to compare NEWS with other early warning systems, which now include complex algorithms derived by logistic regression or machine learning from large electronic medical record (EMR) data sets [
      • Wong A.
      • Otles E.
      • Donnelly J.P.
      • et al.
      External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients.
      ,
      • Churpek M.M.
      • Yuen T.C.
      • Winslow C.
      • et al.
      Multicenter development and validation of a risk stratification tool for ward patients.
      ,
      • Churpek M.M.
      • Yuen T.C.
      • Winslow C.
      • Meltzer D.O.
      • Kattan M.W.
      • Edelson D.P.
      Multicenter comparison of machine learning methods and conventional regression for predicting clinical deterioration on the wards.
      ]. However, transfer to higher levels of care, such as ICU, will vary according to how different hospitals configure their care, as well the number of ICU beds and other resources available. Therefore, to ensure it could compare the performance of NEWS across as many different healthcare settings as possible this review was confined to the prediction of death.
      The aim of this systematic review was to determine and describe the AUC of NEWS and NEWS2 for death at different prediction time windows and/or the absolute mortality above and below defined thresholds or ‘cut-off’ values. The review examined the variation in all the published peer reviewed reports of NEWS and NEWS2 performance, plus additional unpublished data available to the authors. Studies that only reported composite outcomes were excluded.

      2. Methods

      2.1 Study design

      All studies reporting NEWS and NEWS2 with mortality outcome data were reviewed using the Preferred Reporting Items for Systemic Reviews and Meta-Analyses (PRISMA) guidelines [
      • Page M.J.
      • McKenzie J.E.
      • Bossuyt P.M.
      • et al.
      The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.
      ].

      2.2 Eligibility criteria

      All studies reporting NEWS and NEWS2 and providing enough information to determine at different prediction time windows the AUC for death or the absolute mortality above and below defined cut-off values (Table 1). All study designs containing the requisite data were eligible for inclusion, including abstracts. However, studies reporting only composite outcomes were excluded.
      Table 1Inclusion and exclusion criteria.
      SubjectInclusion CriteriaExclusion Criteria
      DateAny timeAny time
      Geographical locationAnyAny
      LanguageEnglishNon-English language, as translation was not feasible
      ParticipantsAdult patientsChildren (age 15 or less)
      Obstetrics
      Peer review orPeer reviewed journalsEditorials
      Type of PublicationPeer reviewed reviewsLetters
      AbstractsNewspaper articles
      Conference abstractsBooks
      Reports, unless peer reviewedTheses
      Dissertations
      Non-peer reviewed articles
      Grey literature
      No mention of ‘National Early Warning Score’ in title or abstract
      Reported outcomesAUC of NEWS and NEWS2 for death at different prediction time windows and/or the absolute mortality above and below defined thresholds or ‘cut-off’ valuesOdds ratios
      Composite outcomes
      SettingAny setting providing data that fulfilled the reported outcomesMaternity
      Obstetrics
      Paediatrics
      Study DesignAny study providing data that fulfilled the reported outcomesStudies with excessive imputed data
      Studies comparing matched samples
      The search was limited to peer reviewed studies reporting adult patients from any speciality, excluding obstetrics. The authors also had access to unpublished supplemental data from two Society for Acute Medicine Benchmarking Audits [
      • Holland M.
      • Subbe C.
      • Atkin C.
      • Knight T.
      • Cooksley T.
      • Lasserson D.
      Society for acute medicine benchmarking audit 2019 (SAMBA19): trends in acute medical care.
      ,
      • Atkin C.
      • Knight T.
      • Subbe C.
      • Holland M.
      • Cooksley T.
      • Lasserson D.
      Acute care service performance during winter: report from the winter SAMBA 2020 national audit of acute care.
      ], and to the data collected from August 2016 to January 2021 by a previously reported ongoing quality improvement project in a low resource Ugandan hospital [
      • Nickel C.H.
      • Kellett J.
      • Nieves Ortega R
      • Lyngholm L.
      • Wasingya-Kasereka L.
      • Brabrand M
      Mobility identifies acutely ill patients at low risk of in-hospital mortality: a prospective multicenter study.
      ].

      2.3 Search and information sources

      The search, which included one paper still in-print [
      • Azijli K.
      • Minderhoud T.
      • Mohammadi P.
      • et al.
      A prospective, observational study of the performance of MEWS, NEWS, SIRS and qSOFA for early risk stratification for adverse outcomes in patients with suspected infections at the emergency department.
      ], was completed on 12 April 2021. [email protected] was used to search MEDLINE, CINHAL and BMJ databases. In addition, PubMed, Paperchase, and the Cochrane Library were searched independently. References in review articles were also manually searched for relevant studies.
      Preliminary searches confirmed that the term ‘NEWS’ was often misinterpreted as a source of information and could not be used. A full text search using the term ‘National Early Warning Score’ identified over 15,000 irrelevant papers. However, when ordered by [email protected] to relevance, limiting the term ‘National Early Warning Score’ to either the abstract or a full text search yielded a comparable number of papers. Therefore, the final search strategy over all search engine platforms was limited to titles and abstracts.
      The search terms used were ‘National Early Warning Score’, ‘Acute’, ‘Emergency’, ‘Mortality’, ‘Survival’, ‘Death’ and ‘Died’. Boolean operators were used to construct the search (Abstract:(National Early Warning Score)) and (Abstract:(National Early Warning Score) AND ((Mortality) OR (Survival) OR (Died) OR (Death)) AND ((Acute OR Emergency)). The search was limited to studies written in English. Although the search was not time limited, NEWS only dates from 2012.

      2.4 Study selection

      Papers were collated using the reference manager ProQuest® RefWorks. Duplicate studies were removed. All titles and abstracts were independently reviewed by both authors. Shortlisted studies were retrieved and independently reviewed by both authors.

      2.5 Data extraction

      Data pertaining to authors, year of publication, country of origin, clinical setting, sample size (number of sets of patient observations, not patients), NEWS, its precursor ViEWS, NEWS2, threshold cut-off values, and mortality were extracted by both authors. Mortality data included the time between an observation and death (i.e., the prediction window).

      2.6 Quality assessment and risk of bias of included studies

      As many studies as possible were included, to ensure that any factors that might bias the performance of NEWS and/or NEWS2 were examined; these included patient age, in-study mortality, study size, location, clinical setting, and the prediction windows and cut-off values reported.

      2.7 Analysis

      Data were extracted to Microsoft Excel (version 2018). Average AUC and mortality rates were calculated for all observations at different prediction windows and cut-off values. AUCs were compared according to the method of Hanley and McNeil [
      • Hanley J.A.
      • McNeil B.J.
      A method of comparing the areas under receiver operating characteristic curves derived from the same cases.
      ]. An AUC of 0.5 suggests no ability to discriminate patients who are going to die from those who will survive, 0.7 to 0.8 is acceptable discrimination, 0.8 to 0.9 is excellent, and more than 0.9 is outstanding [
      • Hosmer D.W.
      • Lemeshow S.
      Applied logistic regression.
      ]

      3. Results

      3.1 Included studies and their characteristics

      The search returned 1323 articles, including 745 duplicates. Of the 578 original studies, 295 were considered eligible for a full review, of which 121 from 28 countries were suitable for inclusion (Fig. 1). Authors to three of these papers [
      • Holland M.
      • Subbe C.
      • Atkin C.
      • Knight T.
      • Cooksley T.
      • Lasserson D.
      Society for acute medicine benchmarking audit 2019 (SAMBA19): trends in acute medical care.
      ,
      • Atkin C.
      • Knight T.
      • Subbe C.
      • Holland M.
      • Cooksley T.
      • Lasserson D.
      Acute care service performance during winter: report from the winter SAMBA 2020 national audit of acute care.
      ,
      • Nickel C.H.
      • Kellett J.
      • Nieves Ortega R
      • Lyngholm L.
      • Wasingya-Kasereka L.
      • Brabrand M
      Mobility identifies acutely ill patients at low risk of in-hospital mortality: a prospective multicenter study.
      ] provided unpublished data, yielding 122 sets of data for analysis. Most were of either consecutive or a convenient sample of patients. One study compared a cross-sectional sample of patients who did not die in hospital with those who died within 24-hours of NEWS measurement [
      • Haegdorens F.
      • Monsieurs K.G.
      • De Meester K.
      • Van Bogaert P.
      The optimal threshold for prompt clinical review: an external validation study of the national early warning score.
      ]. One study [
      • Beane A.
      • De Silva A.P.
      • De Silva N.
      • et al.
      Evaluation of the feasibility and performance of early warning scores to identify patients at risk of adverse outcomes in a low-middle income country setting.
      ] that reported an AUC for 24-hour mortality of 0.65 was excluded from the final analysis because it used imputation so extensively for missing data.
      Fig. 1:
      Fig. 1flow chart of literature search process and eligible studies identified. API = any paper that simply reported and assessment procedure, policy or implementation of an early warning score without outcomes data provided.
      Observations reported ranged from 89 [
      • Shahsavarinia K.
      • Alilou A.M.
      • Alilou S.M.
      • Alilou P.M.
      • Gharekhani A.
      • Gilani N.
      Prognostic accuracy of the quick sequential organ failure assessment score and national early warning scores in mortality rate of the non-traumatic patients.
      ] to 6,222,740 [
      • Pimentel M.A.F.
      • Redfern O.C.
      • Gerry S.
      • et al.
      A comparison of the ability of the National Early Warning Score and the National Early Warning Score 2 to identify patients at risk of in-hospital mortality: a multi-centre database study.
      ], and most were from the UK and the USA (Supplemental Figure 1). Most studies (96 articles) reported the performance of NEWS only or its precursor ViEWS (5 articles); 16 were NEWS2 only, four NEWS and NEWS2, and one NEWS2 and ViEWS; 102 studies only reported the first observation recorded, and only 11 studies reported all observations recorded while in hospital (Supplemental Table 1). Observations were made on patients described as hospital admissions in 38.5% of studies, as patients in emergency departments in 25.4%, emergency admissions in 16.4%, pre-hospital in 10.7%, acute medical unit or medical assessment unit in 4.9%, intensive care units in 2.5%, and as transplant patients in 1.6%. Fifty-one studies included all patients, 22 were limited to patients with sepsis or suspected sepsis, 12 to medical patients only, 11 to patients with pneumonia or respiratory illness, and 10 to patients with COVID-19 or suspected COVID-19 (Table 2). Overall patient age and hospital length of stay could only be approximated as they were not consistently reported in all studies. Citation and further details of the 122 included studies are provided in Supplemental Table 1.
      Table 2Observations made according to where or what type of patient they were recorded on. Observations were made on patient reported as hospital admissions in 37.7% of studies, as in emergency departments in 25.4%, as emergency admissions in 16.4%, as pre-hospital in 10.7%, as in medical assessment units in 5.7%, in intensive care units in 2.5%, and on transplant patients in 1.6%. Fifty-one studies included all patients, 22 were limited to patients with sepsis or suspected sepsis, 12 to medical patients only, 11 to patients with pneumonia or respiratory illness, and 10 to patients with COVID-19 or suspected COVID-19. Other miscellaneous studies were limited to patients specified by illness severity [
      • Mitsunaga T.
      • Hasegawa I.
      • Uzura M.
      • et al.
      Comparison of the National Early Warning Score (NEWS) and the Modified Early Warning Score (MEWS) for predicting admission and in-hospital mortality in elderly patients in the pre-hospital setting and in the emergency department.
      ,
      • Spångfors M.
      • Bunkenborg G.
      • Molt M.
      • Samuelson K.
      The National Early Warning Score predicts mortality in hospital ward patients with deviating vital signs: a retrospective medical record review study.
      ,
      • Chen L.
      • Deng L.
      • Zhao H.
      • et al.
      Comparison of national early warning score, rapid emergency medicine score and acute physiology and chronic health evaluation II score for predicting outcome among emergency severe patients.
      ,
      • Engebretsen S.
      • Bogstrand S.T.
      • Jacobsen D.
      • Vitelli V.
      • Rimstad R.
      NEWS2 versus a single-parameter system to identify critically ill medical patients in the emergency department.
      ,
      • Fernando S.M.
      • Fox-Robichaud A.E.
      • Rochwerg B.
      • et al.
      Prognostic accuracy of the Hamilton Early Warning Score (HEWS) and the National Early Warning Score 2 (NEWS2) among hospitalized patients assessed by a rapid response team.
      ,
      • Graham C.A.
      • Leung L.Y.
      • Lo R.S.L.
      • Yeung C.Y.
      • Chan S.Y.
      • Hung K.K.C.
      NEWS and qSIRS superior to qSOFA in the prediction of 30-day mortality in emergency department patients in Hong Kong.
      ], older age [
      • Dundar Z.D.
      • Kocak S.
      • Girisgin A.S
      Lactate and NEWS-L are fair predictors of mortality in critically ill geriatric emergency department patients.
      ,
      • Kim I.
      • Song H.
      • Kim H.J.
      • et al.
      Use of the National Early Warning Score for predicting in-hospital mortality in older adults admitted to the emergency department.
      ], or specific conditions (i.e., cancer
      • Cooksley T.
      • Kitlowski E.
      • Haji-Michael P.
      Effectiveness of Modified Early Warning Score in predicting outcomes in oncology patients.
      , Lassa fever
      • Duvignaud A.
      • Jaspard M.
      • Etafo I.C.
      • et al.
      LASCOPE study group. Lassa fever outcomes and prognostic factors in Nigeria (LASCOPE): a prospective cohort study.
      , gastrointestinal bleeding
      • Kim D.
      • Jo S.
      • Lee J.B.
      • et al.
      Comparison of the National Early Warning Score+Lactate score with the pre-endoscopic Rockall, Glasgow-Blatchford, and AIMS65 scores in patients with upper gastrointestinal bleeding.
      , acute neurology
      • Zairinal R.A.
      • Kurniawan M.
      Association between the national early warning score and the mortality among neuroemergency patients.
      , tuberculosis
      • Elhidsi M.
      • Rasmin M.
      • Prasenohadi
      In-hospital mortality of pulmonary tuberculosis with acute respiratory failure and related clinical risk factors.
      , stroke
      • Liljehult J.
      • Christensen T.
      Early warning score predicts acute mortality in stroke patients.
      , Klebsiella infection
      • Wang J.L.
      • Lu X.Y.
      • Xu X.H.
      • et al.
      Predictive role of monocyte-to-lymphocyte ratio in patients with Klebsiella pneumonia infection: a single-center experience.
      , and pancreatitis
      • Tan J.W.
      • Zhang X.Q.
      • Geng C.M.
      • Peng L.L.
      Development of the national early warning score-calcium model for predicting adverse outcomes in patients with acute pancreatitis.
      ). AMU/MAU = acute medical or medical assessment unit. PRE-HOSPITAL = prior to hospital transfer, recorded before entering or in the ambulance en route to hospital.
      PATIENTS INCLUDEDHOSPITAL ADMISSIONSEMERGENCY DEPARTMENTEMERGENCY ADMISSIONSPRE-HOSPITALAMU/MAUICUTransplant patientsTotal
      All patients20941303251 (41.8%)
      Suspected sepsis4125010022 (18.0%)
      Medical only610050012 (9.8%)
      Pneumonia or respiratory820010011 (9.0%)
      Suspected COVID-19127000010 (8.2%)
      Miscellaneous754000016 (13.1%)
      Total46 (37.7%)31 (25.4%)20 (16.4%)13 (10.7%)7 (5.7%)3 (2.5%)2 (1.6%)122 (100.0%)

      3.2 Performance of news across different prediction windows

      The discrimination of NEWS for death has been reported for several prediction windows. We found 74 published reports and one unpublished report from which the average AUC could be estimated, ranging from 89 to 2,759,469 observations (Supplemental Table 2a). The total number of observations for prediction windows ranged from over 5 million for 24-hour mortality, 1,888,535 for in hospital death and 101,680 for death within 30-days. The overall AUC averages decline from 0.897 at 24-hours, to 0.762 by 30 days. The range of AUC reported by each study increased with the length of the prediction window, ranging from 0.801 – 0.910 for 24-hour mortality to 0.610 – 0.910 for 30-day mortality (Fig. 2). Similar declines in AUC for death as the time after observation increased were observed for NEWS2 (Supplemental Table 2b) and ViEWS (Supplemental Table 2c).
      Fig. 2:
      Fig. 2The area under the Receiver Operating Characteristic curve (AUC) for mortality at different times (i.e., predictions windows up to 30 days) after NEWS recorded. Results of individual studies are shown as open circles, solid triangles show average result for each prediction window if more than one study was performed. Complete data is provided in Supplemental Table 2a; data for NEWS2 and ViEWS in Supplemental Tables 2b and 2c.

      3.3 Comparison of auc for 24-hour mortality of news with news 2 and views (Table 3)

      One unpublished and 15 published studies reported the discrimination of NEWS for death within 24-hours; the average AUC for a total of 5,637,622 observations was 0.897 (95% CI 0.895 – 0.899; range 0.801 to 0.910). Five papers reported that 269,008 measurements of ViEWS had an average AUC for death within 24-hours of 0.886 (95% CI 0.877 – 0.895, range 0.873 – 0.908), and three papers reported an average AUC of 0.848 (95% CI 0.811 – 0.885, range 0.840 – 0.861) for 9717 measurements of NEWS 2 (Table 3). The smaller the number of observations made, the greater the range of AUC reported; all studies with more than 50,000 observations had an AUC ≥0.880 (Supplemental Figure 2).
      Table 3the area under the Receiver Operating Characteristic curve (AUC) for 24-hour mortality from all eligible studies for NEWS, NEWS2 and ViEWS. Overall results of approximate average age, total number of observations and average AUC are shown in bold. 95% CI = 95% confidence interval.
      EWSReferenceAge (years)CountryObservation made on or in:Number observations24 -hour mortalityAUC95%CI
      NEWSSmith et al. 2016
      • Smith G.B.
      • Prytherch D.R.
      • Jarvis S.
      • et al.
      A comparison of the ability of the physiologic components of medical emergency team criteria and the U.K. National early warning score to discriminate patients at risk of a range of adverse clinical outcomes.
      62.2 SD 20.4UKHOSPITAL ADMISSIONS: all patients2,245,7780.50%0.9100.9100.92
      NEWSUnpublished data Uganda

      Unpublished data provided by the Kitovu Hospital Study Group 2021.

      48 (IQR 28–68)UGANDAHOSPITAL ADMISSIONS: all patients58880.30%0.9010.8040.999
      NEWSEndo et al. 2020
      • Endo T.
      • Yoshida T.
      • Shinozaki T.
      • et al.
      Efficacy of prehospital National Early Warning Score to predict outpatient disposition at an emergency department of a Japanese tertiary hospital: a retrospective study.
      73 (IQR 53–82)JAPANPRE-HOSPITAL: all patients28470.80%0.9000.8700.930
      NEWSSmith et al. 2013
      • Smith G.B.
      • Prytherch D.R.
      • Meredith P.
      • Schmidt P.E.
      • Featherstone P.I.
      The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death.
      73 (median)UKHOSPITAL ADMISSION: all patients198,7550.90%0.8940.8870.902
      NEWSFaisal et al. 2019
      • Faisal M.
      • Richardson D.
      • Scally A.
      • Howes R.
      • Beatson K.
      • Mohammed M.
      Performance of externally validated enhanced computer-aided versions of the National Early Warning Score in predicting mortality following an emergency admission to hospital in England: a cross-sectional study.
      67.1 SD 19.5UKEMERGENCY ADMISSION: all patients35,8070.60%0.8930.8710.914
      NEWSDelahanty et al. 2019
      • Delahanty R.J.
      • Alvarez J.
      • Flynn L.M.
      • Sherwin R.L.
      • Jones S.S.
      Development and evaluation of a machine learning model for the early identification of patients at risk for sepsis.
      48.00 SD 20.14USAEMERGENCY DEPARTMENT: sepsis?2,759,4690.60%0.8900.8900.890
      NEWSLee et al. 2020
      • Lee S.B.
      • Kim D.H.
      • Kim T.
      • et al.
      Emergency Department Triage Early Warning Score (TREWS) predicts in-hospital mortality in the emergency department.
      64 (IQR 50–75)KOREAEMERGENCY ADMISSION: all patients27,1732.90%0.8840.8800.888
      NEWSCampbell et al. 2020
      • Campbell V.
      • Conway R.
      • Carey K.
      • et al.
      Predicting clinical deterioration with Q-ADDS compared to NEWS, Between the Flags, and eCART track and trigger tools.
      57 (IQR 41–69)USAHOSPITAL ADMISSION: all patients224,9121.00%0.8800.8800.890
      NEWSLee et al. 2020
      • Lee S.B.
      • Kim D.H.
      • Kim T.
      • et al.
      Emergency Department Triage Early Warning Score (TREWS) predicts in-hospital mortality in the emergency department.
      64 (IQR 50–75)KOREAEMERGENCY ADMISSION: all patients54,3473.00%0.8780.8750.881
      NEWSFaisal et al. 2019
      • Faisal M.
      • Richardson D.
      • Scally A.
      • Howes R.
      • Beatson K.
      • Mohammed M.
      Performance of externally validated enhanced computer-aided versions of the National Early Warning Score in predicting mortality following an emergency admission to hospital in England: a cross-sectional study.
      67.1 SD 19.5UKEMERGENCY ADMISSION: all patients35,1610.50%0.8640.8350.894
      NEWSSilcock et al. 2015
      • Silcock D.J.
      • Corfield A.R.
      • Gowens P.A.
      • Rooney K.D.
      Validation of the national early warning score in the prehospital setting.
      ≥16UKPRE-HOSPITAL: all patients1,6841.00%0.8550.6901.000
      NEWSLiu et al. 2015
      • Liu F.Y.
      • Qin J.
      • Wang R.X.
      • et al.
      A prospective validation of National Early Warning Score in emergency intensive care unit patients at Beijing.
      62.2 SD 18CHINAEMERGENCY ICU: all patients54010.20%0.8500.7900.900
      NEWSPirneskoski et al. 2019
      • Pirneskoski J.
      • Kuisma M.
      • Olkkola K.T.
      • Nurmi J.
      Prehospital National Early Warning Score predicts early mortality.
      66.0 SD 20.0FINLANDPRE-HOSPITAL: all patients35,8801.10%0.8400.8230.858
      NEWSRichardson et al. 2021
      • Richardson D.
      • Faisal M.
      • Fiori M.
      • Beatson K.
      • Mohammed M.
      Use of the first National Early Warning Score recorded within 24 h of admission to estimate the risk of in-hospital mortality in unplanned COVID-19 patients: a retrospective cohort study.
      73.3SD 15.4UKEMERGENCY ADMISSION: non COVID-195,8240.90%0.8400.7800.980
      NEWSRichardson et al. 2021
      • Richardson D.
      • Faisal M.
      • Fiori M.
      • Beatson K.
      • Mohammed M.
      Use of the first National Early Warning Score recorded within 24 h of admission to estimate the risk of in-hospital mortality in unplanned COVID-19 patients: a retrospective cohort study.
      73.3SD 15.4UKEMERGENCY ADMISSION: COVID-19?6201.50%0.8400.700.990
      NEWSBrabrand et al. 2017
      • Brabrand M.
      • Hallas P.
      • Hansen S.N.
      • Jensen K.M.
      • Madsen J.L.B.
      • Posth S.
      Using scores to identify patients at risk of short term mortality at arrival to the acute medical unit: a validation study of six existing scores.
      67 (range 49–78)DENMARKHOSPITAL ADMISSION: MAU2,7080.60%0.8300.6401.000
      NEWSHaegdorens et al. 2020
      • Haegdorens F.
      • Monsieurs K.G.
      • De Meester K.
      • Van Bogaert P.
      The optimal threshold for prompt clinical review: an external validation study of the national early warning score.
      64.8 SD 14.5BELGIUMHOSPITAL ADMISSION: matched sample1,9133.40%0.8010.7660.837
      Overall≠55.55,639,3060.60%0.8970.8950.899
      NEWS2Martin-Rodriguez et al. 2020c
      • Martín-Rodríguez F.
      • Sanz-García A.
      • Medina-Lozano E.
      • et al.
      The Value of prehospital early warning scores to predict in - hospital clinical deterioration: a multicenter, observational base-ambulance study.
      69 years (IQR 54–81)SPAINPRE-HOSPITAL: all patients3,2733.50%0.8610.8100.900
      NEWS2Richardson et al. 2021
      • Richardson D.
      • Faisal M.
      • Fiori M.
      • Beatson K.
      • Mohammed M.
      Use of the first National Early Warning Score recorded within 24 h of admission to estimate the risk of in-hospital mortality in unplanned COVID-19 patients: a retrospective cohort study.
      67.7 SD 19.0UKEMERGENCY ADMISSION: COVID-19?6201.50%0.8600.7500.970
      NEWS2Richardson et al. 2021
      • Richardson D.
      • Faisal M.
      • Fiori M.
      • Beatson K.
      • Mohammed M.
      Use of the first National Early Warning Score recorded within 24 h of admission to estimate the risk of in-hospital mortality in unplanned COVID-19 patients: a retrospective cohort study.
      67.7 SD 19.0UKEMERGENCY ADMISSION: non-COVID-195,8240.90%0.8400.7800.900
      Overall≠68.19,7171.80%0.8480.8110.885
      ViEWSKellett et al. 2013
      • Kellett J.
      • Clifford M.
      • Ridley A.
      • Gleeson M
      Validation of the VitalPACTM early warning score (ViEWS) in acutely ill medical patients admitted.
      not statedIRELANDAMBULATORY CARE UNIT: medical3,1171.20%0.9080.8420.973
      ViEWSPrytherch et al. 2010
      • Prytherch D.R.
      • Smith G.B.
      • Schmidt P.E.
      • Featherstone P.I.
      ViEWS—Towards a national early warning score for detecting adult inpatient deterioration.
      67.7UKHOSPITAL ADMISSION: all patients198,7551.00%0.8880.8800.895
      ViEWSOpio et al. 2013
      • Opio M.O.
      • Nansubuga G.
      • Kellett J.
      Validation of the VitalPACTM Early Warning Score (ViEWS) in acutely ill medical patients attending a resource-poor hospital in sub-Saharan Africa.
      42 (IQR 26–65)UGANDAHOSPITAL ADMISSION: medical4,2201.60%0.8860.8250.947
      ViEWSChurpek et al. 2013
      • Churpek M.M.
      • Yuen T.C.
      • Edelson D.P.
      Risk stratification of hospitalized patients on the wards.
      55 SD 18USAHOSPITAL ADMISSION: all patients59,6430.50%0.8800.8600.910
      ViEWSMartin-Rodriguez et al. 2020c
      • Martín-Rodríguez F.
      • Sanz-García A.
      • Medina-Lozano E.
      • et al.
      The Value of prehospital early warning scores to predict in - hospital clinical deterioration: a multicenter, observational base-ambulance study.
      69 years (IQR 54–81)SPAINPRE-HOSPITAL: all patients3,2733.50%0.8730.8100.900
      Overall≠64.5269,0080.90%0.8860.8770.895

      3.4 Potential modifiers of the AUC of news for 24-hour mortality (Table 4)

      Most observations of NEWS were made either on UK patients throughout their hospital stay (2,245,778 observations) or in emergency departments patients in the USA (2,759,469), but the AUC for 24-hour mortality was little influenced by where, geographically, it was measured (Table 4). However, the AUC of NEWS for 24-hour mortality was lower in studies with a small number of observations and only changed slightly by patient age. The AUC for 24-hour mortality of observations made throughout hospitalization are slightly higher than for 24-hour mortality after the first observation made. The 24-hour mortality AUC may be lower if observations are made prior to hospital admission, or in the emergency department, or in patients admitted as emergencies or in ICU, compared with observations made on any patient admitted to hospital. The overall 24-hour mortality of patients with NEWS observations ranged from 0.3% to 10.2%, and the AUC for 24-hour mortality decreased slightly as 24-hour mortality increased. The average AUC for 24-hour mortality reported for observations on patients with a 24-hour mortality ≤0.6% was 0.910 (95% CI 0.906 – 0.914), compared with 0.866 (95% CI 0.857 – 0.875) for observations on patients with a 24-hour mortality ≥1% (Table 4).
      Table 4Potential modifiers of the AUC of NEWS for 24-hour mortality. AUC = the area under the Receiver Operating Characteristic curve (AUC) for 24-hour mortality, 95% CI = 95% confidence interval.
      VariableReferenceObservations24-hour mortalityAUC95%CI95%CI
      Geographic location
      Asia32, 52, 5384,9072.94%0.8800.8710.889
      Africa605,8880.30%0.9010.8040.998
      Europe23, 33, 39, 48, 56–58, 612,564,1300.54%0.9070.9040.910
      America49, 512,984,3810.63%0.8890.8860.892
      Number observations
      ≥2 million51, 575,005,2470.56%0.8990.8970.901
      190,000–230,00049, 58423,6670.95%0.8870.8800.893
      27,000–55,00033, 39, 52188,3681.70%0.8720.8640.880
      ≤600023, 32, 48, 53, 56, 6022,0241.16%0.8610.8310.890
      Age
      ≤50 years51, 602,765,3570.60%0.8900.8870.893
      50–70 years23, 33, 39, 48, 52, 56, 572,670,0430.63%0.9040.9010.908
      ≥70 years32, 56, 58202,2220.90%0.8940.8840.904
      Index observation
      Observations throughout hospitalization49, 57, 5826,69,4450.57%0.9060.9030.909
      First observation only32, 33, 39, 48, 51, 52, 56, 60, 6130,21,7550.71%0.8890.8860.892
      Clinical site of observation
      Pre-hospital32, 33, 48, 6143,1191.04%0.8440.8210.867
      Emergency department or admission39, 51, 52, 53, 562,918,9410.67%0.8890.8860.892
      Any hospital admission23, 49, 57, 58, 602,677,2460.57%0.9060.9030.909
      24-hour mortality
      ≤0.6%39, 58, 602,286,8270.50%0.9090.9050.913
      0.6%−1.0%32, 39, 48, 49, 51, 56, 58, 613,232,0060.65%0.8890.8860.892
      ≥1%23, 33, 52, 53, 56120,4732.44%0.8660.8580.875

      3.5 The relationship of AUC of news to in-hospital mortality and underlying illness (Table 5)

      After studies that reported NEWS AUC for 24-hour mortality, the next largest set of observations (1,890,093) were from studies that reported AUC for in-hospital mortality. Only 15 of these 35 studies also reported hospital length of stay with averages ranging from 2.6 to 20 days (Supplemental Table 2a). Although the length of hospital stays could only be surmised, the AUC for in-hospital mortality decreased as the absolute in-hospital mortality observed increased. Also, the AUC for in-hospital mortality of patients with respiratory illness, sepsis or suspected sepsis was significantly lower (p <0.0001) than for other patients (Table 5)
      Table 5The AUC of NEWS for in-hospital mortality according to the absolute in-hospital mortality observed and patients’ underlying illness. AUC = the area under the Receiver Operating Characteristic curve (AUC) for in-hospital mortality, 95% CI = 95% confidence interval. ED = emergency department, ICU = intensive care unit.
      VariableReferenceObservationsIn-hospital mortalityAUC95%CI95%CI
      In-hospital mortality
      ≤2%
      • Kivipuro M.
      • Tirkkonen J.
      • Kontula T.
      • et al.
      National early warning score (NEWS) in a Finnish multidisciplinary emergency department and direct vs. late admission to intensive care.
      ,
      • Liu V.X.
      • Lu Y.
      • Carey K.A.
      • et al.
      Comparison of early warning scoring systems for hospitalized patients with and without infection at risk for in-hospital mortality and transfer to the intensive care unit.
      ,
      • Nieves Ortega R
      • Rosin C.
      • Bingisser R.
      • Nickel C.H
      Clinical scores and formal triage for screening of sepsis and adverse outcomes on arrival in an emergency department all-comer cohort.
      ,
      • Usman O.A.
      • Usman A.A.
      • Ward M.A.
      Comparison of SIRS, qSOFA, and NEWS for the early identification of sepsis in the Emergency Department.
      833,3971.49%0.8630.8580.867
      2–4%
      • Jo S.
      • Yoon J.
      • Lee J.B.
      • Jin Y.
      • Jeong T.
      • Park B.
      Predictive value of the National Early Warning Score-Lactate for mortality and the need for critical care among general emergency department patients.
      ,
      • Kim D.
      • Jo S.
      • Lee J.B.
      • et al.
      Comparison of the National Early Warning Score+Lactate score with the pre-endoscopic Rockall, Glasgow-Blatchford, and AIMS65 scores in patients with upper gastrointestinal bleeding.
      ,
      • Lane D.J.
      • Wunsch H.
      • Saskin R.
      • et al.
      Assessing Severity of Illness in Patients Transported to Hospital by Paramedics: external Validation of 3 Prognostic Scores.
      ,
      • Lee Y.S.
      • Choi J.W.
      • Park Y.H.
      • et al.
      Evaluation of the efficacy of the National Early Warning Score in predicting in-hospital mortality via the risk stratification.
      ,
      • Liu V.X.
      • Lu Y.
      • Carey K.A.
      • et al.
      Comparison of early warning scoring systems for hospitalized patients with and without infection at risk for in-hospital mortality and transfer to the intensive care unit.
      ,
      • Tan J.W.
      • Zhang X.Q.
      • Geng C.M.
      • Peng L.L.
      Development of the national early warning score-calcium model for predicting adverse outcomes in patients with acute pancreatitis.
      ,
      • Wattanasit P.
      • Khwannimit B.
      Comparison the accuracy of early warning scores with qSOFA and SIRS for predicting sepsis in the emergency department.
      ,
      • Liu F.
      • Qin J.
      • Liang X.
      • Wang J.
      • Chan P.
      Comparison of four risk scoring systems for old patients: a prospective multicenter study in China.
      904,4522.99%0.8570.8540.860
      4–6%
      • Mitsunaga T.
      • Hasegawa I.
      • Uzura M.
      • et al.
      Comparison of the National Early Warning Score (NEWS) and the Modified Early Warning Score (MEWS) for predicting admission and in-hospital mortality in elderly patients in the pre-hospital setting and in the emergency department.
      ,
      • Faisal M.
      • Richardson D.
      • Scally A.
      • Howes R.
      • Beatson K.
      • Mohammed M.
      Performance of externally validated enhanced computer-aided versions of the National Early Warning Score in predicting mortality following an emergency admission to hospital in England: a cross-sectional study.
      ,
      • Hodgson L.E.
      • Dimitrov B.D.
      • Congleton J.
      • Venn R.
      • Forni L.G.
      • Roderick P.J.
      A validation of the National Early Warning Score to predict outcome in patients with COPD exacerbation.
      ,
      • Richardson D.
      • Faisal M.
      • Fiori M.
      • Beatson K.
      • Mohammed M.
      Use of the first National Early Warning Score recorded within 24 h of admission to estimate the risk of in-hospital mortality in unplanned COVID-19 patients: a retrospective cohort study.
      ,
      • Kim I.
      • Song H.
      • Kim H.J.
      • et al.
      Use of the National Early Warning Score for predicting in-hospital mortality in older adults admitted to the emergency department.
      ,
      • de Groot B.
      • Stolwijk F.
      • Warmerdam M.
      • et al.
      The most commonly used disease severity scores are inappropriate for risk stratification of older emergency department sepsis patients: an observational multi-centre study.
      123,1325.43%0.7480.7410.755
      ≥6%
      • Shahsavarinia K.
      • Alilou A.M.
      • Alilou S.M.
      • Alilou P.M.
      • Gharekhani A.
      • Gilani N.
      Prognostic accuracy of the quick sequential organ failure assessment score and national early warning scores in mortality rate of the non-traumatic patients.
      ,
      • Osawa I.
      • Sonoo T.
      • Soeno S.
      • Hara K.
      • Nakamura K.
      • Goto T.
      Clinical performance of early warning scoring systems for identifying sepsis among anti-hypertensive agent users.
      ,
      • Richardson D.
      • Faisal M.
      • Fiori M.
      • Beatson K.
      • Mohammed M.
      Use of the first National Early Warning Score recorded within 24 h of admission to estimate the risk of in-hospital mortality in unplanned COVID-19 patients: a retrospective cohort study.
      ,
      • Baker T.
      • Blixt J.
      • Lugazia E.
      • et al.
      Single deranged physiologic parameters are associated with mortality in a low-income country.
      ,
      • Covino M.
      • De Matteis G.
      • Burzo M.L.
      • et al.
      GEMELLI AGAINST COVID-19 group. Predicting in-hospital mortality in COVID-19 older patients with specifically developed scores.
      ,
      • Dundar Z.D.
      • Kocak S.
      • Girisgin A.S
      Lactate and NEWS-L are fair predictors of mortality in critically ill geriatric emergency department patients.
      ,
      • Echevarria C.
      • Steer J.
      • Bourke S.C.
      Comparison of early warning scores in patients with COPD exacerbation: DECAF and NEWS score.
      ,
      • Goulden R.
      • Hoyle M.C.
      • Monis J.
      • et al.
      qSOFA, SIRS and NEWS for predicting inhospital mortality and ICU admission in emergency admissions treated as sepsis.
      ,
      • Hu H.
      • Yao N.
      • Qiu Y.
      Predictive value of 5 early warning scores for critical COVID-19 patients.
      ,
      • Jo S.
      • Jeong T.
      • Lee J.B.
      • Jin Y.
      • Yoon J.
      • Park B.
      Validation of modified early warning score using serum lactate level in community-acquired pneumonia patients. The National Early Warning Score-Lactate score.
      ,
      • Liu F.Y.
      • Sun X.L.
      • Zhang Y.
      • et al.
      Evaluation of the risk prediction tools for patients with coronavirus disease 2019 in wuhan, china: a single-centered, retrospective, observational study.
      ,
      • Phungoen P.
      • Khemtong S.
      • Apiratwarakul K.
      • Ienghong K.
      • Kotruchin P.
      Emergency Severity Index as a predictor of in-hospital mortality in suspected sepsis patients in the emergency department.
      ,
      • Ruangsomboon O.
      • Boonmee P.
      • Limsuwat C.
      • Chakorn T.
      • Monsomboon A.
      The utility of the rapid emergency medicine score (REMS) compared with SIRS, qSOFA and NEWS for Predicting in-hospital Mortality among Patients with suspicion of Sepsis in an emergency department.
      ,
      • Spångfors M.
      • Bunkenborg G.
      • Molt M.
      • Samuelson K.
      The National Early Warning Score predicts mortality in hospital ward patients with deviating vital signs: a retrospective medical record review study.
      ,
      • Suresh S.
      • Tiwari A.
      • Mathew R.
      • et al.
      Predictors of mortality and the need of mechanical ventilation in confirmed COVID-19 patients presenting to the emergency department in North India.
      ,
      • Zairinal R.A.
      • Kurniawan M.
      Association between the national early warning score and the mortality among neuroemergency patients.
      ,
      • de Groot B.
      • Stolwijk F.
      • Warmerdam M.
      • et al.
      The most commonly used disease severity scores are inappropriate for risk stratification of older emergency department sepsis patients: an observational multi-centre study.
      23,32511.34%0.7140.7020.725
      Diagnoses
      Respiratory pneumonia43, 65, 685,5597.37%0.6720.6420.702
      Sepsis44, 66, 78, 79, 84, 8617,5749.06%0.7180.7030.733
      COVID-1956, 63, 67, 75, 811,93823.32%0.7710.7430.799
      Remaining patients reported as:
      a). pre-hospital, ED, or ICU25, 35, 39, 56, 62, 64, 69, 71–73, 77, 82, 83, 85, 87331,9263.14%0.8090.8040.814
      b). hospital admissions43, 70, 74, 76, 801,527,3092.35%0.8620.8600.865

      3.6 Range of absolute 24-hour mortalities observed at different news cut-off values (Table 6)

      One unpublished and 12 published studies provided enough information to determine the absolute mortality within 24-hours for different cut-off values of NEWS, from which the 24-hour mortality rate for patients with values above and below the cut-off could be determined. Although the ranges of 24-hour mortality of patients with NEWS points greater or equal to the cut-off are wide (Table 6), their average values trend upwards from 0.8% for a cut-off ≥1 point to 7.8% for a cut-off ≥7 points (Fig. 3). In contrast, the 24-hour mortality for patients with NEWS values below all cut-off values only increased from 0.04% for a cut-off ≥1 point to 0.23% for a cut-off ≥7 points (Table 6). Although patients with NEWS <7 points only had a 24-hour mortality of 0.23%, these deaths accounted for 44.3% of deaths within 24-hours, whereas the deaths of patients with a NEWS <3 points made up 8.9% of all 24-hour deaths (Fig. 4).
      Table 6Range of absolute 24-hour mortalities observed at different NEWS cut-off values. Overall results of approximate average age, total number of observations and average values are shown in bold. IQR = inter-quartile range. SD = standard deviations.
      ReferenceAge (years)CountryObservation made on or in:Cut-offTotal observationsPatientsMortality ≥ cut-offMortality ≤cut-off% Deaths ≤cut-offAll patient
      ≥ cut-off24 hr mortality
      Vihonen et al. 2020
      • Vihonen H.
      • Lääperi M.
      • Kuisma M.
      • Pirneskoski J.
      • Nurmi J.
      Glucose as an additional parameter to National Early Warning Score (NEWS) in prehospital setting enhances identification of patients at risk of death: an observational cohort study.
      69 (IQR 52–81)FINLANDPRE-HOSPITAL≥127,14182%0.84%0.04%1.10%0.70%
      Lee et al. 2020
      • Lee S.B.
      • Kim D.H.
      • Kim T.
      • et al.
      Emergency Department Triage Early Warning Score (TREWS) predicts in-hospital mortality in the emergency department.
      64 (IQR 50–75)KOREAEMERGENCY ADMISSIONS≥354,34750%5.83%0.30%5.00%3.00%
      Smith et al. 2016
      • Smith G.B.
      • Prytherch D.R.
      • Jarvis S.
      • et al.
      A comparison of the ability of the physiologic components of medical emergency team criteria and the U.K. National early warning score to discriminate patients at risk of a range of adverse clinical outcomes.
      62.2 SD 20.4UKHOSPITAL ADMISSION - all patients≥32,245,77827%1.62%0.06%9.50%0.50%
      Unpublished data Uganda

      Unpublished data provided by the Kitovu Hospital Study Group 2021.

      48 (IQR 28–68)UGANDAHOSPITAL ADMISSION - all patients≥35,88875%0.39%0.00%0.00%0.30%
      ≠ 62.232,306,01327%1.80%0.07%8.90%0.50%
      Smith et al. 2016
      • Smith G.B.
      • Prytherch D.R.
      • Jarvis S.
      • et al.
      A comparison of the ability of the physiologic components of medical emergency team criteria and the U.K. National early warning score to discriminate patients at risk of a range of adverse clinical outcomes.
      62.2 SD 20.4UKHOSPITAL ADMISSION - all patients≥42,245,77815%2.58%0.10%17.10%0.50%
      Vihonen et al. 2020
      • Vihonen H.
      • Lääperi M.
      • Kuisma M.
      • Pirneskoski J.
      • Nurmi J.
      Glucose as an additional parameter to National Early Warning Score (NEWS) in prehospital setting enhances identification of patients at risk of death: an observational cohort study.
      69 (IQR 52–81)FINLANDPRE-HOSPITAL≥427,14139%1.60%0.12%10.00%0.70%
      ≠ 62.342,272,91916%2.55%0.10%17.00%0.50%
      Atmaca et al. 2018
      • Atmaca Ö.
      • Turan C.
      • Güven P.
      • Arıkan H.
      • Eryüksel S.E.
      • Karakurt S.
      Usage of NEWS for prediction of mortality and in-hospital cardiac arrest rates in a Turkish university hospital.
      56 (range 19 - 94)TURKEYHOSPITAL ADMISSION - medical≥510419%20.00%1.19%20.00%4.80%
      Endo et al. 2020
      • Endo T.
      • Yoshida T.
      • Shinozaki T.
      • et al.
      Efficacy of prehospital National Early Warning Score to predict outpatient disposition at an emergency department of a Japanese tertiary hospital: a retrospective study.
      73 (IQR 53–82)JAPANPRE-HOSPITAL≥52,84753%1.45%0.00%0.00%0.80%
      Faisal et al. 2019
      • Faisal M.
      • Richardson D.
      • Scally A.
      • Howes R.
      • Beatson K.
      • Mohammed M.
      Performance of externally validated enhanced computer-aided versions of the National Early Warning Score in predicting mortality following an emergency admission to hospital in England: a cross-sectional study.
      67.1 SD 19.5UKEMERGENCY ADMISSIONS≥535,80718%2.79%0.13%17.10%0.60%
      Faisal et al. 2019
      • Faisal M.
      • Richardson D.
      • Scally A.
      • Howes R.
      • Beatson K.
      • Mohammed M.
      Performance of externally validated enhanced computer-aided versions of the National Early Warning Score in predicting mortality following an emergency admission to hospital in England: a cross-sectional study.
      67.1 SD 19.5UKEMERGENCY ADMISSIONS≥535,16114%2.64%0.18%28.80%0.50%
      Forster et al. 2018
      • Forster S.
      • Housley G.
      • McKeever T.M.
      • Shaw D.E.
      Investigating the discriminative value of Early Warning Scores in patients with respiratory disease using a retrospective cohort analysis of admissions to Nottingham University Hospitals Trust over a 2-year period.
      63.1UKHOSPITAL ADMISSION – respiratory≥5236,8409%1.98%0.02%11.30%0.20%
      Haegdorens et al. 2020
      • Haegdorens F.
      • Monsieurs K.G.
      • De Meester K.
      • Van Bogaert P.
      The optimal threshold for prompt clinical review: an external validation study of the national early warning score.
      64.8 SD 14.5BELGIUMHOSPITAL ADMISSION – all patients≥51,91323%11.01%1.09%24.60%3.40%
      Hoikka et al. 2018
      • Hoikka M.
      • Silfvast T.
      • TI Ala-Kokko
      Does the prehospital National Early Warning Score predict the short-term mortality of unselected emergency patients?.
      65.4 SD 20.0FINLANDPRE-HOSPITAL≥512,42621%6.67%0.21%11.00%1.50%
      Pimentel et al. 2019
      • Pimentel M.A.F.
      • Redfern O.C.
      • Gerry S.
      • et al.
      A comparison of the ability of the National Early Warning Score and the National Early Warning Score 2 to identify patients at risk of in-hospital mortality: a multi-centre database study.
      68 (IQR 50–80)UKHOSPITAL ADMISSION – all patients≥56,229,7409%0.91%0.03%26.60%0.10%
      Richardson et al. 2021
      • Richardson D.
      • Faisal M.
      • Fiori M.
      • Beatson K.
      • Mohammed M.
      Use of the first National Early Warning Score recorded within 24 h of admission to estimate the risk of in-hospital mortality in unplanned COVID-19 patients: a retrospective cohort study.
      67.7 SD 19.0UKEMERGENCY ADMISSIONS≥55,82412%4.53%0.41%39.60%0.90%
      Richardson et al. 2021
      • Richardson D.
      • Faisal M.
      • Fiori M.
      • Beatson K.
      • Mohammed M.
      Use of the first National Early Warning Score recorded within 24 h of admission to estimate the risk of in-hospital mortality in unplanned COVID-19 patients: a retrospective cohort study.
      73.3SD 15.4UKEMERGENCY ADMISSSIONS - COVID-19≥562030%4.37%0.23%11.10%1.50%
      Smith et al. 2016
      • Smith G.B.
      • Prytherch D.R.
      • Jarvis S.
      • et al.
      A comparison of the ability of the physiologic components of medical emergency team criteria and the U.K. National early warning score to discriminate patients at risk of a range of adverse clinical outcomes.
      62.2 SD 20.4UKHOSPITAL ADMISSION - all patients≥52,245,7789%3.95%0.13%24.80%0.50%
      ≠ 66.458,807,0609%1.78%0.06%24.90%0.20%
      Smith et al. 201662.2 SD 20.4UKHOSPITAL ADMISSION - all patients≥62,245,7785%5.93%0.18%34.60%0.50%
      Hoikka et al. 2018
      • Hoikka M.
      • Silfvast T.
      • TI Ala-Kokko
      Does the prehospital National Early Warning Score predict the short-term mortality of unselected emergency patients?.
      65.4 SD 20.0FINLANDPRE-HOSPITAL≥712,4266%21.31%0.32%19.90%1.50%
      Pirneskoski et al. 2019
      • Pirneskoski J.
      • Kuisma M.
      • Olkkola K.T.
      • Nurmi J.
      Prehospital National Early Warning Score predicts early mortality.
      66.0 SD 20.0FINLANDPRE-HOSPITAL≥735,88023%3.46%0.32%23.00%1.10%
      Smith et al. 2016
      • Smith G.B.
      • Prytherch D.R.
      • Jarvis S.
      • et al.
      A comparison of the ability of the physiologic components of medical emergency team criteria and the U.K. National early warning score to discriminate patients at risk of a range of adverse clinical outcomes.
      62.2 SD 20.4UKHOSPITAL ADMISSION - all patients≥72,245,7783%8.53%0.23%45.80%0.50%
      Vihonen et al. 2020
      • Vihonen H.
      • Lääperi M.
      • Kuisma M.
      • Pirneskoski J.
      • Nurmi J.
      Glucose as an additional parameter to National Early Warning Score (NEWS) in prehospital setting enhances identification of patients at risk of death: an observational cohort study.
      69 (IQR 52–81)FINLANDPRE-HOSPITAL≥727,14117%2.86%0.25%28.90%0.70%
      ≠ 62.472,321,2254%7.78%0.23%44.30%0.50%
      Fig. 3:
      Fig. 324-hour mortality above and below NEWS cut-off thresholds ranging from 1 to 7 points. Note that the most frequently used cut-off value reported in the literature is ≥5 points. Full data is shown in .
      Fig. 4:
      Fig. 4The proportion of all deaths within 24-hours of patients below NEWS cut-off thresholds ranging from 1 to 7 points, and the proportion of all patients equal or above each cut-off. Although patients with <7 NEWS points only had a 24-hour mortality of 0.23%, these deaths accounted for 44.3% of deaths within 24-hours, whereas the deaths of patients with a NEWS <3 points made up 8.9% of all 24-hour deaths.

      3.7 Absolute mortalities observed at different news cut-off values and prediction windows

      The mortality for patients at different time intervals after an observation for NEWS cut-off values ≥1 to ≥15 points were determined from 24-hours to 30 days and at hospital discharge. The range of mortality rates above cut-off values varied greatly, but less for mortality rates below each cut-off (Supplemental Tables 3a-3 g). The most frequently reported cut-off values were ≥3, ≥5 and ≥7. Within 30 days 9.4% of patient with a NEWS ≥3, 14.9% with a NEWS ≥5 and 20.6% with a NEWS ≥7 points had died (Fig. 5). In contrast, for patients with NEWS <3 points, the average mortality remained below 1% throughout their hospital stay, and for 72 h for patients with a NEWS <5; the mortality of patients with a NEWS <7 remained below 2% for up to 5 days. Less than 4% of patients had died within 30 days of a NEWS observation of <7 points. Nevertheless, 21.8% of all deaths within 30 days of an observation occurred in patients with a NEWS <3, 51.9% in patients with a NEWS <5, and 74.5% in patient with a NEWS <7 points.
      Fig. 5:
      Fig. 5Mortality at different times after NEWS observation above and below cut-off values of 3, 5 and 7 points.

      4. Discussion

      4.1 General interpretation of the results in the context of other evidence

      The overall risk of death within 24-hours for most patients reported is low, and NEWS reliably identifies those patients who are the least likely to die within 24-hours. After 24-hours the discrimination of NEWS for subsequent death declines, with an ever-widening range of reported results, which by 30 days after a NEWS observation is from an AUC of 0.61 to 0.91. Although patients with low NEWS scores make up a significant fraction of all in-hospital deaths, these patients appear to have a reduced risk of death for some time, which implies they have a degree of clinical stability.
      This review found that patients with a NEWS <3 points, on average, only have a 0.07% chance of dying within 24-hours. It has been suggested that measuring a complete set of vital signs in these patients more frequently than once a day is not required [
      • van Galen L.S.
      • Dijkstra C.C.
      • Ludikhuize J.
      • Kramer M.H.
      • Nanayakkara P.W.
      A protocolised once a day modified early warning score (MEWS) measurement is an appropriate screening tool for major adverse events in a general hospital population.
      ]. However, these “low risk” patients still need some form of ongoing monitoring as they comprise 8.9% of all deaths within 24-hours. The best way to anticipate deterioration in these patients remains unclear. Unlike patients with NEWS <4 points, all patients with a NEWS <3 points will have normal mental status. Nearly three quarters of all patients were observed to have a NEWS <3 points, and these patients accounted for 9% of all deaths within 24-hours and 16% of all in-hospital deaths. No study reported a mortality above 0.35% within 10 days of a NEWS <3 being observed, and in most studies their risk of in-hospital death remained below 1%. However, one study in Thailand of patients with suspected sepsis [
      • Boonmee P.
      • Ruangsomboon O.
      • Limsuwat C.
      • Chakorn T.
      Predictors of mortality in elderly and very elderly emergency patients with sepsis: a retrospective study.
      ] reported patients with a NEWS <3 points had an in-hospital mortality ranging from 12% to 21%. Therefore, although there may be specific patient cohorts with normal or near normal vital signs who are likely to die while in hospital, very few of them are likely to die within 24-hours.
      The discrimination of NEWS was not significantly influenced by patient age or the country it was measured in. However, it did decline as in-hospital mortality rates increased, as has been observed with another EWS [
      • Prytherch D.R.
      • Smith G.B.
      • Schmidt P.E.
      • Featherstone P.I.
      ViEWS—Towards a national early warning score for detecting adult inpatient deterioration.
      ]. Moreover, although several studies have reported the performance of NEWS in the pre-hospital setting [
      • Alam N.
      • Vegting I.L.
      • Houben E.
      • et al.
      Exploring the performance of the National Early Warning Score (NEWS) in a European emergency department.
      ,
      • Hargreaves D.S.
      • de Carvalho J.L.J.
      • Smith L.
      • Picton G.
      • Venn R.
      • Hodgson L.E.
      Persistently elevated early warning scores and lactate identifies patients at high risk of mortality in suspected sepsis.
      ,
      • Inada-Kim M.
      • Knight T.
      • Sullivan M.
      • et al.
      The prognostic value of national early warning scores (NEWS) during transfer of care from community settings to hospital: a retrospective service evaluation.
      ,
      • Williams T.A.
      • Tohira H.
      • Finn J.
      • Perkins G.D.
      • Ho K.M.
      The ability of early warning scores (EWS) to detect critical illness in the prehospital setting: a systematic review.
      ] with conflicting results, this review found that the sooner NEWS is measured the lower its AUC for 24-hour mortality [
      • Endo T.
      • Yoshida T.
      • Shinozaki T.
      • et al.
      Efficacy of prehospital National Early Warning Score to predict outpatient disposition at an emergency department of a Japanese tertiary hospital: a retrospective study.
      ,
      • Pirneskoski J.
      • Kuisma M.
      • Olkkola K.T.
      • Nurmi J.
      Prehospital National Early Warning Score predicts early mortality.
      ]. A possible explanation for both these observations may be the availability of life-saving interventions and when and where they were or could be provided. Early measurements were probably made before any treatment was provided, whereas later measurements were more likely after everything possible had been done, and little more could save the patient. This hypothesis is supported by the report of Hwang et al. [
      • Hwang T.S.
      • Park H.W.
      • Park H.Y.
      • Park Y.S.
      Prognostic value of severity score change for septic shock in the emergency room.
      ] who found that the AUC of NEWS for 7-day mortality of patients with sepsis was 0.680 when measured at triage, 0.777 after resuscitation with fluid and vasopressors, and 0.888 on exiting the emergency department. Others have also shown the discrimination of NEWS to be lower on arrival to hospital than when repeated later [
      • Alam N.
      • Vegting I.L.
      • Houben E.
      • et al.
      Exploring the performance of the National Early Warning Score (NEWS) in a European emergency department.
      ,
      • Mitsunaga T.
      • Hasegawa I.
      • Uzura M.
      • et al.
      Comparison of the National Early Warning Score (NEWS) and the Modified Early Warning Score (MEWS) for predicting admission and in-hospital mortality in elderly patients in the pre-hospital setting and in the emergency department.
      ]. Similarly, our finding that the discrimination of NEWS2 for 24-hour mortality is marginally inferior to NEWS should be viewed with caution because only three NEWS2 studies of 9,717 patients fulfilled our inclusion criteria, and all reported emergency patients early in their treatment. Therefore, although NEWS or NEWS2 may not accurately predict the mortality of conditions for which there are effective treatments, such as sepsis [
      • Hamilton F.
      • Arnold D.
      • Baird A.
      • Albur M.
      • Whiting P.
      Early Warning Scores do not accurately predict mortality in sepsis: a meta-analysis and systematic review of the literature.
      ], this does not mean that they cannot identify patients who need to be treated urgently for them.

      4.2 Limitations of the review processes used

      The CHARM checklist for systematic reviews of predictive models [
      • Moons K.G.
      • de Groot J.A.
      • Bouwmeester W.
      • et al.
      Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist.
      ] was not adhered to as the aim of this review was to determine from all the available literature, biased or otherwise, the range of discrimination for mortality and the mortality rates associated with different levels of NEWS at different prediction windows, in different patient populations. It intentionally only examined absolute mortality and did not consider relative risks or odds ratios and did not distinguish between those deaths that were preventable and those that were not. Our literature search was confined to papers on NEWS or NEWS2 and did not specifically search for ViEWS. For pragmatic reasons the final search strategy over all search engine platforms was limited to titles and abstracts. It is possible that papers that otherwise fulfilled our inclusion criteria but only referred to NEWS in the body of their text were missed. As our search ended on April 12th, 2021, it only included publications reporting the performance of NEWS in COVID-19 patients published up to that date.

      4.3 Limitations of the evidence included in the review

      The information available made it difficult to determine and compare the age and length of hospital stay for patients in different studies. Although most studies validated NEWS at a single point of time, commonly using the first observation recorded [
      • Fang A.H.S.
      • Lim W.T.
      • Balakrishnan T.
      Early warning score validation methodologies and performance metrics: a systematic review.
      ], treatments given before the observation were poorly documented. We could only determine patient diagnoses in those studies where they were specified, and we have no information on the quality of care provided. As authors often used the same data in different publications, we made every effort to ensure that the results for AUC, specific prediction windows and cut-off values were only used once.
      NEWS requires the accurate measurement of vital signs and correct calculation [
      • Prytherch D.R.
      • Smith G.B.
      • Schmidt P.
      • et al.
      Calculating early warning scores–a classroom comparison of pen and paper and hand-held computer methods.
      ]. It is recorded and calculated more faithfully when entered contemporaneously into an electronic system than on paper [
      • Faisal M.
      • Richardson D.
      • Scally A.
      • Howes R.
      • Beatson K.
      • Mohammed M.
      Performance of externally validated enhanced computer-aided versions of the National Early Warning Score in predicting mortality following an emergency admission to hospital in England: a cross-sectional study.
      ]; only 13 studies (10.7%) explicitly stated that the vital sign measurements to calculate NEWS were recorded electronically and contemporaneously. Moreover, the measurement of respiratory rate is often influenced by subjective bias [
      • Churpek M.M.
      • Snyder A.
      • Twu N.M.
      • Edelson D.P.
      Accuracy comparisons between manual and automated respiratory rate for detecting clinical deterioration in ward patients.
      ], while the assessment of mental status [
      • Brunker C.
      • Harris R.
      How accurate is the AVPU scale in detecting neurological impairment when used by general ward nurses? An evaluation study using simulation and a questionnaire.
      ] and the need for supplemental oxygen is subjective and may vary according to opinion, expertise and the availability of oxygen and other resources in different locations and clinical settings [
      • Wasingya-Kasereka L.
      • Nabatanzi P.
      • Nakitende I.
      • Nabiryo J.
      • Namujwiga T.
      • Kellett J.
      Kitovu Hospital Study Group. Oxygen use in low-resource settings: an intervention still triggered by intuition.
      ]. The discrimination of NEWS may be lower in COPD patients [
      • Hodgson L.E.
      • Dimitrov B.D.
      • Congleton J.
      • Venn R.
      • Forni L.G.
      • Roderick P.J.
      A validation of the National Early Warning Score to predict outcome in patients with COPD exacerbation.
      ], and medication may also change NEWS performance; the AUC of NEWS for in-hospital mortality was lower in patients with suspected sepsis on hypertensive medication than those not on medication [
      • Osawa I.
      • Sonoo T.
      • Soeno S.
      • Hara K.
      • Nakamura K.
      • Goto T.
      Clinical performance of early warning scoring systems for identifying sepsis among anti-hypertensive agent users.
      ]. Despite these and other possible unconsidered confounders, we found only two studies that reported an AUC for 24-hour mortality <0.83, one used extensive imputation for missing data and was excluded [
      • Beane A.
      • De Silva A.P.
      • De Silva N.
      • et al.
      Evaluation of the feasibility and performance of early warning scores to identify patients at risk of adverse outcomes in a low-middle income country setting.
      ], and the other selected patients by sampling instead of consecutive observations [
      • Haegdorens F.
      • Monsieurs K.G.
      • De Meester K.
      • Van Bogaert P.
      The optimal threshold for prompt clinical review: an external validation study of the national early warning score.
      ].

      4.4 Implications of the results for practice, policy, and future research

      Whilst the risk of death increases as the cut-off increases (Fig. 3), the optimal point for intervention is unclear. The available literature we reviewed shows that if a cut-off of ≥7 points is selected, only 4% of patients would trigger an intervention and 44% of patients who die will be missed. Alternatively, a cut-off of ≥1 point will trigger an assessment and/or an intervention in 83% of patients, which may not benefit many of them. NEWS ≥5 points is the most adopted cut-off and 91% of patients are observed to have a NEWS below it; the overall 24-hour mortality of these patients <5 points is only 0.06%, but their in-hospital mortality averaged nearly 3% (range 0.65% to 24.7%). Moreover, a quarter of all deaths within 24-hours and more than 40% of all in-hospital deaths occur in patients with a NEWS <5. Although NEWS was never intended to be used as a diagnostic test for any specific condition, NEWS ≥5 has been recommended as a flag for sepsis [
      • Silcock D.J.
      • Corfield A.R.
      • Staines H.
      • Rooney K.D.
      Superior performance of National Early Warning Score compared with quick Sepsis-related Organ Failure Assessment Score in predicting adverse outcomes: a retrospective observational study of patients in the prehospital setting.
      ]. However, this NEWS value might not be the optimal level to start antibiotics [
      • Ferrer R.
      • Martin-Loeches I.
      • Phillips G.
      • et al.
      Empiric antibiotic treatment reduces mortality in severe sepsis and septic shock from the first hour: results from a guideline-based performance improvement program.
      ] or other time-sensitive interventions [
      • Ferrer R.
      • Martin-Loeches I.
      • Phillips G.
      • et al.
      Empiric antibiotic treatment reduces mortality in severe sepsis and septic shock from the first hour: results from a guideline-based performance improvement program.
      ,
      • Keep J.W.
      • Messmer A.S.
      • Sladden R.
      • et al.
      National early warning score at Emergency Department triage may allow earlier identification of patients with severe sepsis and septic shock: a retrospective observational study.
      ]. Moreover, many life-saving interventions, such as anti-coagulation for pulmonary embolus, thrombolysis for stroke, emergency surgery, rehydration to prevent acute kidney injury etc., should be given as soon as possible, and regardless of the patient's NEWS value.

      5. Conclusion

      NEWS reliably discriminates between patients who are most and least likely to die within 24-hours. It can, therefore, reliably identify sick patients who may need urgent attention, which is what it was designed to do. However, after 24-hours the prediction of mortality by NEWS declines and becomes unreliable.
      The predictive performance of NEWS is likely to be lower if it is recorded before an effective treatment is delivered, and will be highest if no treatment is required, or there is no effective treatment, or if everything possible has already been done. Although NEWS or NEWS2 may be less likely to accurately predict the mortality of patients given urgent effective treatments, this does not mean that they cannot accurately identify patients who need them.
      Although patients with a low NEWS score appear to have a reduced risk of death for several days, implying a degree of clinical and physiological stability, many of them die while still in hospital. The best way to anticipate deterioration in these patients remains unclear.

      Authors’ contributions

      All authors contributed to the preparation of this paper. MH and JK conceived the study and supervised the collection of the data and ensured its accuracy, analysed the data, and drafted the manuscript and critically revised the manuscript for intellectual content. Both authors read and approved the final manuscript and are guarantors of the paper.

      Ethical approval

      As there was no patient involvement ethical approval was not required for this study.

      Declaration of Competing Interest

      All costs were borne by the authors. John Kellett is a major shareholder, director, and chief medical officer of Tapa Healthcare DAC. Mark Holland has no potential conflicts of interest.

      Acknowledgments

      The authors wish to acknowledge access to anonymised unpublished data provided by the Kitovu Hospital Study Group, Kitovu Hospital, Masaka, Uganda, and the organising committee of the Society for Acute Medicine Benchmarking Audit (SAMBA).

      Appendix. Supplementary materials

      References

        • Morgan R.J.
        • Wright M.M.
        In defence of early warning scores.
        Br J Anaesth. 2007; 99: 747-748
        • Gerry S.
        • Bonnici T.
        • Birks J.
        • et al.
        Early warning scores for detecting deterioration in adult hospital patients: systematic review and critical appraisal of methodology.
        BMJ. 2020; 369: m1501https://doi.org/10.1136/bmj.m1501
        • Wright M.M.
        • Morgan R.J.M
        In defence of early warning scores.
        BMJ. 2020; 369 (Available at) ((Accessed 16 May 2010]): m1501
        • Mandrekar J.N.
        Receiver operating characteristic curve in diagnostic test assessment.
        J Thorac Oncol. 2010; 5: 1315-1316
        • Royal College of Physicians
        National Early Warning Score (NEWS): standardising the assessment of acute illness severity in the NHS.
        Report of a working party. RCP, London2012 (Accessed 9 October 2021, at file:///C:/Users/markd/Downloads/National%20Early%20Warning%20Score%20(NEWS)%20-%20Standardising%20the%20assessment%20of%20acute-illness%20severity%20in%20the%20NHS_0%20(2).pdf)
        • Dean J.D.
        • Williams B.
        • Inada-Kim M.
        Rapid Response: early warning scores supplement clinical judgement.
        BMJ. 2020; 369 (Accessed 9 October 2021, at https://www.bmj.com/content/369/bmj.m1501/rr-09)
        • Prytherch D.R.
        • Smith G.B.
        • Schmidt P.E.
        • Featherstone P.I.
        ViEWS—Towards a national early warning score for detecting adult inpatient deterioration.
        Resuscitation. 2010; 81: 932-937
        • Royal College of Physicians
        National Early Warning Score (NEWS) 2. Standardising the assessment of acute illness severity in the NHS.
        Report of a working party. 19th. RCP, LondonDecember 2017 (last accessed December 15th 2021)
        • Pimentel M.A.F.
        • Redfern O.C.
        • Gerry S.
        • et al.
        A comparison of the ability of the National Early Warning Score and the National Early Warning Score 2 to identify patients at risk of in-hospital mortality: a multi-centre database study.
        Resuscitation. 2019; 134: 147-156
        • O’Driscoll R.
        • Bakerly N.
        • Murphy P.
        • Turkington P.
        NEWS2 needs to be tested in prospective trials involving patients with confirmed hypercapnia.
        Resuscitation. 2019; 139: 369-370
        • Hodgson L.E.
        • Congleton J.
        • Venn R.
        • Forni L.G.
        • Roderick P.J.
        NEWS 2 – too little evidence to implement?.
        Clin Med (Lond). 2018 Oct; 18; : 371-373
        • Fang A.H.S.
        • Lim W.T.
        • Balakrishnan T.
        Early warning score validation methodologies and performance metrics: a systematic review.
        BMC Med Inform Decis Mak. 2020; 20: 111-118https://doi.org/10.1186/s12911-020-01144-8
        • Wong A.
        • Otles E.
        • Donnelly J.P.
        • et al.
        External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients.
        JAMA Intern Med. 2021; 181: 1065-1070
        • Churpek M.M.
        • Yuen T.C.
        • Winslow C.
        • et al.
        Multicenter development and validation of a risk stratification tool for ward patients.
        Am J Respir Crit Care Med. 2014; 190: 649-655
        • Churpek M.M.
        • Yuen T.C.
        • Winslow C.
        • Meltzer D.O.
        • Kattan M.W.
        • Edelson D.P.
        Multicenter comparison of machine learning methods and conventional regression for predicting clinical deterioration on the wards.
        Crit Care Med. 2016; 44: 368-374
        • Page M.J.
        • McKenzie J.E.
        • Bossuyt P.M.
        • et al.
        The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.
        BMJ. 2021; 372: n71https://doi.org/10.1136/bmj.n71
        • Holland M.
        • Subbe C.
        • Atkin C.
        • Knight T.
        • Cooksley T.
        • Lasserson D.
        Society for acute medicine benchmarking audit 2019 (SAMBA19): trends in acute medical care.
        Acute Med. 2020; 19: 209-219
        • Atkin C.
        • Knight T.
        • Subbe C.
        • Holland M.
        • Cooksley T.
        • Lasserson D.
        Acute care service performance during winter: report from the winter SAMBA 2020 national audit of acute care.
        Acute Med. 2020; 19: 220-229
        • Nickel C.H.
        • Kellett J.
        • Nieves Ortega R
        • Lyngholm L.
        • Wasingya-Kasereka L.
        • Brabrand M
        Mobility identifies acutely ill patients at low risk of in-hospital mortality: a prospective multicenter study.
        Chest. 2019; 156: 316-322
        • Azijli K.
        • Minderhoud T.
        • Mohammadi P.
        • et al.
        A prospective, observational study of the performance of MEWS, NEWS, SIRS and qSOFA for early risk stratification for adverse outcomes in patients with suspected infections at the emergency department.
        Acute Med. 2021; 20: 116-124
        • Hanley J.A.
        • McNeil B.J.
        A method of comparing the areas under receiver operating characteristic curves derived from the same cases.
        Radiology. 1983; 148: 839-843
        • Hosmer D.W.
        • Lemeshow S.
        Applied logistic regression.
        2nd ed. John Wiley and Sons, New York2000: 160-164
        • Haegdorens F.
        • Monsieurs K.G.
        • De Meester K.
        • Van Bogaert P.
        The optimal threshold for prompt clinical review: an external validation study of the national early warning score.
        J Clin Nurs. 2020; 29: 4594-4603
        • Beane A.
        • De Silva A.P.
        • De Silva N.
        • et al.
        Evaluation of the feasibility and performance of early warning scores to identify patients at risk of adverse outcomes in a low-middle income country setting.
        BMJ Open. 2018; 8e019387https://doi.org/10.1136/bmjopen-2017-019387
        • Shahsavarinia K.
        • Alilou A.M.
        • Alilou S.M.
        • Alilou P.M.
        • Gharekhani A.
        • Gilani N.
        Prognostic accuracy of the quick sequential organ failure assessment score and national early warning scores in mortality rate of the non-traumatic patients.
        Arch Anesth & Crit Care. 2021; 7: 7-11https://doi.org/10.18502/aacc.v7i1.5474
        • van Galen L.S.
        • Dijkstra C.C.
        • Ludikhuize J.
        • Kramer M.H.
        • Nanayakkara P.W.
        A protocolised once a day modified early warning score (MEWS) measurement is an appropriate screening tool for major adverse events in a general hospital population.
        PLoS ONE. 2016; 11e0160811https://doi.org/10.1371/journal.pone.0160811
        • Boonmee P.
        • Ruangsomboon O.
        • Limsuwat C.
        • Chakorn T.
        Predictors of mortality in elderly and very elderly emergency patients with sepsis: a retrospective study.
        West J Emerg Med. 2020; 21: 210-218https://doi.org/10.5811/westjem.2020.7.47405
        • Alam N.
        • Vegting I.L.
        • Houben E.
        • et al.
        Exploring the performance of the National Early Warning Score (NEWS) in a European emergency department.
        Resuscitation. 2015; 90: 111-115
        • Hargreaves D.S.
        • de Carvalho J.L.J.
        • Smith L.
        • Picton G.
        • Venn R.
        • Hodgson L.E.
        Persistently elevated early warning scores and lactate identifies patients at high risk of mortality in suspected sepsis.
        Eur J Emerg Med. 2020; 27: 125-131
        • Inada-Kim M.
        • Knight T.
        • Sullivan M.
        • et al.
        The prognostic value of national early warning scores (NEWS) during transfer of care from community settings to hospital: a retrospective service evaluation.
        BJGP Open. 2020; 4https://doi.org/10.3399/bjgpopen20X101071
        • Williams T.A.
        • Tohira H.
        • Finn J.
        • Perkins G.D.
        • Ho K.M.
        The ability of early warning scores (EWS) to detect critical illness in the prehospital setting: a systematic review.
        Resuscitation. 2016; 102: 35-43
        • Endo T.
        • Yoshida T.
        • Shinozaki T.
        • et al.
        Efficacy of prehospital National Early Warning Score to predict outpatient disposition at an emergency department of a Japanese tertiary hospital: a retrospective study.
        BMJ Open. 2020; 10e034602https://doi.org/10.1136/bmjopen-2019-034602
        • Pirneskoski J.
        • Kuisma M.
        • Olkkola K.T.
        • Nurmi J.
        Prehospital National Early Warning Score predicts early mortality.
        Acta Anaesthesiol Scand. 2019; 63: 676-683
        • Hwang T.S.
        • Park H.W.
        • Park H.Y.
        • Park Y.S.
        Prognostic value of severity score change for septic shock in the emergency room.
        Diagnostics (Basel). 2020; 10: 743https://doi.org/10.3390/diagnostics10100743
        • Mitsunaga T.
        • Hasegawa I.
        • Uzura M.
        • et al.
        Comparison of the National Early Warning Score (NEWS) and the Modified Early Warning Score (MEWS) for predicting admission and in-hospital mortality in elderly patients in the pre-hospital setting and in the emergency department.
        PeerJ. 2019; 7: e6947https://doi.org/10.7717/peerj.6947
        • Hamilton F.
        • Arnold D.
        • Baird A.
        • Albur M.
        • Whiting P.
        Early Warning Scores do not accurately predict mortality in sepsis: a meta-analysis and systematic review of the literature.
        J Infect. 2018; 76: 241-248
        • Moons K.G.
        • de Groot J.A.
        • Bouwmeester W.
        • et al.
        Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist.
        PLoS Med. 2014; 11e1001744https://doi.org/10.1371/journal.pmed.1001744
        • Prytherch D.R.
        • Smith G.B.
        • Schmidt P.
        • et al.
        Calculating early warning scores–a classroom comparison of pen and paper and hand-held computer methods.
        Resuscitation. 2006; 70: 173-178
        • Faisal M.
        • Richardson D.
        • Scally A.
        • Howes R.
        • Beatson K.
        • Mohammed M.
        Performance of externally validated enhanced computer-aided versions of the National Early Warning Score in predicting mortality following an emergency admission to hospital in England: a cross-sectional study.
        BMJ Open. 2019; 9e031596https://doi.org/10.1136/bmjopen-2019-031596
        • Churpek M.M.
        • Snyder A.
        • Twu N.M.
        • Edelson D.P.
        Accuracy comparisons between manual and automated respiratory rate for detecting clinical deterioration in ward patients.
        J Hosp Med. 2018; 13: 486-487
        • Brunker C.
        • Harris R.
        How accurate is the AVPU scale in detecting neurological impairment when used by general ward nurses? An evaluation study using simulation and a questionnaire.
        Intensive Crit Care Nurs. 2015; 31: 69-75
        • Wasingya-Kasereka L.
        • Nabatanzi P.
        • Nakitende I.
        • Nabiryo J.
        • Namujwiga T.
        • Kellett J.
        Kitovu Hospital Study Group. Oxygen use in low-resource settings: an intervention still triggered by intuition.
        Resusc Plus. 2020; 4100056https://doi.org/10.1016/j.resplu.2020.100056
        • Hodgson L.E.
        • Dimitrov B.D.
        • Congleton J.
        • Venn R.
        • Forni L.G.
        • Roderick P.J.
        A validation of the National Early Warning Score to predict outcome in patients with COPD exacerbation.
        Thorax. 2017; 72: 23-30
        • Osawa I.
        • Sonoo T.
        • Soeno S.
        • Hara K.
        • Nakamura K.
        • Goto T.
        Clinical performance of early warning scoring systems for identifying sepsis among anti-hypertensive agent users.
        Am J Emerg Med. 2021; 48: 120-127
        • Silcock D.J.
        • Corfield A.R.
        • Staines H.
        • Rooney K.D.
        Superior performance of National Early Warning Score compared with quick Sepsis-related Organ Failure Assessment Score in predicting adverse outcomes: a retrospective observational study of patients in the prehospital setting.
        Eur J Emerg Med. 2019; 26: 433-439
        • Ferrer R.
        • Martin-Loeches I.
        • Phillips G.
        • et al.
        Empiric antibiotic treatment reduces mortality in severe sepsis and septic shock from the first hour: results from a guideline-based performance improvement program.
        Crit Care Med. 2014; 42: 1749-1755
        • Keep J.W.
        • Messmer A.S.
        • Sladden R.
        • et al.
        National early warning score at Emergency Department triage may allow earlier identification of patients with severe sepsis and septic shock: a retrospective observational study.
        Emerg Med J. 2016; 33: 37-41
        • Brabrand M.
        • Hallas P.
        • Hansen S.N.
        • Jensen K.M.
        • Madsen J.L.B.
        • Posth S.
        Using scores to identify patients at risk of short term mortality at arrival to the acute medical unit: a validation study of six existing scores.
        Eur J Intern Med. 2017; 45: 32-36
        • Campbell V.
        • Conway R.
        • Carey K.
        • et al.
        Predicting clinical deterioration with Q-ADDS compared to NEWS, Between the Flags, and eCART track and trigger tools.
        Resuscitation. 2020; 153: 28-34
        • Churpek M.M.
        • Yuen T.C.
        • Edelson D.P.
        Risk stratification of hospitalized patients on the wards.
        Chest. 2013; 143: 1758-1765
        • Delahanty R.J.
        • Alvarez J.
        • Flynn L.M.
        • Sherwin R.L.
        • Jones S.S.
        Development and evaluation of a machine learning model for the early identification of patients at risk for sepsis.
        Ann Emerg Med. 2019; 73: 334-344
        • Lee S.B.
        • Kim D.H.
        • Kim T.
        • et al.
        Emergency Department Triage Early Warning Score (TREWS) predicts in-hospital mortality in the emergency department.
        Am J Emerg Med. 2020; 38: 203-210
        • Liu F.Y.
        • Qin J.
        • Wang R.X.
        • et al.
        A prospective validation of National Early Warning Score in emergency intensive care unit patients at Beijing.
        Hong Kong J Emergency Med. 2015; 22: 137-144https://doi.org/10.1177/102490791502200302
        • Martín-Rodríguez F.
        • Sanz-García A.
        • Medina-Lozano E.
        • et al.
        The Value of prehospital early warning scores to predict in - hospital clinical deterioration: a multicenter, observational base-ambulance study.
        Prehosp Emerg Care. 2020; : 1-10https://doi.org/10.1080/10903127.2020.1813224
        • Opio M.O.
        • Nansubuga G.
        • Kellett J.
        Validation of the VitalPACTM Early Warning Score (ViEWS) in acutely ill medical patients attending a resource-poor hospital in sub-Saharan Africa.
        Resuscitation. 2013; 84: 743-746
        • Richardson D.
        • Faisal M.
        • Fiori M.
        • Beatson K.
        • Mohammed M.
        Use of the first National Early Warning Score recorded within 24 h of admission to estimate the risk of in-hospital mortality in unplanned COVID-19 patients: a retrospective cohort study.
        BMJ Open. 2021; 11e043721https://doi.org/10.1136/bmjopen-2020-043721
        • Smith G.B.
        • Prytherch D.R.
        • Jarvis S.
        • et al.
        A comparison of the ability of the physiologic components of medical emergency team criteria and the U.K. National early warning score to discriminate patients at risk of a range of adverse clinical outcomes.
        Crit Care Med. 2016; 44: 2171-2181
        • Smith G.B.
        • Prytherch D.R.
        • Meredith P.
        • Schmidt P.E.
        • Featherstone P.I.
        The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death.
        Resuscitation. 2013; 84: 465-470
        • Kellett J.
        • Clifford M.
        • Ridley A.
        • Gleeson M
        Validation of the VitalPACTM early warning score (ViEWS) in acutely ill medical patients admitted.
        Ir Med J. 2013; 106: 318
      1. Unpublished data provided by the Kitovu Hospital Study Group 2021.

        • Silcock D.J.
        • Corfield A.R.
        • Gowens P.A.
        • Rooney K.D.
        Validation of the national early warning score in the prehospital setting.
        Resuscitation. 2015; 89: 31-35
        • Baker T.
        • Blixt J.
        • Lugazia E.
        • et al.
        Single deranged physiologic parameters are associated with mortality in a low-income country.
        Crit Care Med. 2015; 43: 2171-2179
        • Covino M.
        • De Matteis G.
        • Burzo M.L.
        • et al.
        GEMELLI AGAINST COVID-19 group. Predicting in-hospital mortality in COVID-19 older patients with specifically developed scores.
        J Am Geriatr Soc. 2021; 69: 37-43
        • Dundar Z.D.
        • Kocak S.
        • Girisgin A.S
        Lactate and NEWS-L are fair predictors of mortality in critically ill geriatric emergency department patients.
        Am J Emerg Med. 2020; 38: 217-221
        • Echevarria C.
        • Steer J.
        • Bourke S.C.
        Comparison of early warning scores in patients with COPD exacerbation: DECAF and NEWS score.
        Thorax. 2019; 74: 941-946
        • Goulden R.
        • Hoyle M.C.
        • Monis J.
        • et al.
        qSOFA, SIRS and NEWS for predicting inhospital mortality and ICU admission in emergency admissions treated as sepsis.
        Emerg Med J. 2018; 35: 345-349
        • Hu H.
        • Yao N.
        • Qiu Y.
        Predictive value of 5 early warning scores for critical COVID-19 patients.
        Disaster Med Public Health Prep. 2020; : 1-8https://doi.org/10.1017/dmp.2020.324
        • Jo S.
        • Jeong T.
        • Lee J.B.
        • Jin Y.
        • Yoon J.
        • Park B.
        Validation of modified early warning score using serum lactate level in community-acquired pneumonia patients. The National Early Warning Score-Lactate score.
        Am J Emerg Med. 2016; 34: 536-541
        • Jo S.
        • Yoon J.
        • Lee J.B.
        • Jin Y.
        • Jeong T.
        • Park B.
        Predictive value of the National Early Warning Score-Lactate for mortality and the need for critical care among general emergency department patients.
        J Crit Care. 2016; 36: 60-68
        • Kim D.
        • Jo S.
        • Lee J.B.
        • et al.
        Comparison of the National Early Warning Score+Lactate score with the pre-endoscopic Rockall, Glasgow-Blatchford, and AIMS65 scores in patients with upper gastrointestinal bleeding.
        Clin Exp Emerg Med. 2018; 5: 219-229
        • Kim I.
        • Song H.
        • Kim H.J.
        • et al.
        Use of the National Early Warning Score for predicting in-hospital mortality in older adults admitted to the emergency department.
        Clin Exp Emerg Med. 2020; 7: 61-66
        • Kivipuro M.
        • Tirkkonen J.
        • Kontula T.
        • et al.
        National early warning score (NEWS) in a Finnish multidisciplinary emergency department and direct vs. late admission to intensive care.
        Resuscitation. 2018; 128: 164-169
        • Lane D.J.
        • Wunsch H.
        • Saskin R.
        • et al.
        Assessing Severity of Illness in Patients Transported to Hospital by Paramedics: external Validation of 3 Prognostic Scores.
        Prehosp Emerg Care. 2020; 24: 273-281
        • Lee Y.S.
        • Choi J.W.
        • Park Y.H.
        • et al.
        Evaluation of the efficacy of the National Early Warning Score in predicting in-hospital mortality via the risk stratification.
        J Crit Care. 2018; 47: 222-226
        • Liu F.Y.
        • Sun X.L.
        • Zhang Y.
        • et al.
        Evaluation of the risk prediction tools for patients with coronavirus disease 2019 in wuhan, china: a single-centered, retrospective, observational study.
        Crit Care Med. 2020; 48: e1004-e1011https://doi.org/10.1097/CCM.0000000000004549
        • Liu V.X.
        • Lu Y.
        • Carey K.A.
        • et al.
        Comparison of early warning scoring systems for hospitalized patients with and without infection at risk for in-hospital mortality and transfer to the intensive care unit.
        JAMA Netw Open. 2020; 3e205191https://doi.org/10.1001/jamanetworkopen.2020.5191
        • Nieves Ortega R
        • Rosin C.
        • Bingisser R.
        • Nickel C.H
        Clinical scores and formal triage for screening of sepsis and adverse outcomes on arrival in an emergency department all-comer cohort.
        J Emerg Med. 2019; 57: 453-460https://doi.org/10.1016/j.jemermed.2019.06.036
        • Phungoen P.
        • Khemtong S.
        • Apiratwarakul K.
        • Ienghong K.
        • Kotruchin P.
        Emergency Severity Index as a predictor of in-hospital mortality in suspected sepsis patients in the emergency department.
        Am J Emerg Med. 2020; 38: 1854-1859
        • Ruangsomboon O.
        • Boonmee P.
        • Limsuwat C.
        • Chakorn T.
        • Monsomboon A.
        The utility of the rapid emergency medicine score (REMS) compared with SIRS, qSOFA and NEWS for Predicting in-hospital Mortality among Patients with suspicion of Sepsis in an emergency department.
        BMC Emerg Med. 2021 Jan 7; 21: 2https://doi.org/10.1186/s12873-020-00396-x
        • Spångfors M.
        • Bunkenborg G.
        • Molt M.
        • Samuelson K.
        The National Early Warning Score predicts mortality in hospital ward patients with deviating vital signs: a retrospective medical record review study.
        J Clin Nurs. 2019; 28: 1216-1222
        • Suresh S.
        • Tiwari A.
        • Mathew R.
        • et al.
        Predictors of mortality and the need of mechanical ventilation in confirmed COVID-19 patients presenting to the emergency department in North India.
        J Family Med Prim Care. 2021; 10: 542-549https://doi.org/10.4103/jfmpc.jfmpc_1775_20
        • Tan J.W.
        • Zhang X.Q.
        • Geng C.M.
        • Peng L.L.
        Development of the national early warning score-calcium model for predicting adverse outcomes in patients with acute pancreatitis.
        J Emerg Nurs. 2020; 46: 171-179
        • Usman O.A.
        • Usman A.A.
        • Ward M.A.
        Comparison of SIRS, qSOFA, and NEWS for the early identification of sepsis in the Emergency Department.
        Am J Emerg Med. 2019; 37: 1490-1497
        • Wattanasit P.
        • Khwannimit B.
        Comparison the accuracy of early warning scores with qSOFA and SIRS for predicting sepsis in the emergency department.
        Am J Emerg Med. 2021; 46: 284-288
        • Zairinal R.A.
        • Kurniawan M.
        Association between the national early warning score and the mortality among neuroemergency patients.
        J Nat Sc Biol Med. 2019; 10: S16-S19https://doi.org/10.4103/jnsbm.JNSBM_87_19
        • de Groot B.
        • Stolwijk F.
        • Warmerdam M.
        • et al.
        The most commonly used disease severity scores are inappropriate for risk stratification of older emergency department sepsis patients: an observational multi-centre study.
        Scand J Trauma Resusc Emerg Med. 2017; 25: 91https://doi.org/10.1186/s13049-017-0436-3
        • Liu F.
        • Qin J.
        • Liang X.
        • Wang J.
        • Chan P.
        Comparison of four risk scoring systems for old patients: a prospective multicenter study in China.
        Innov Aging. 2017; 1 (PMCID: PMC6246620): 598https://doi.org/10.1093/geroni/igx004.2094
        • Atmaca Ö.
        • Turan C.
        • Güven P.
        • Arıkan H.
        • Eryüksel S.E.
        • Karakurt S.
        Usage of NEWS for prediction of mortality and in-hospital cardiac arrest rates in a Turkish university hospital.
        Turk J Med Sci. 2018; 48: 1087-1091https://doi.org/10.3906/sag-1706-67
        • Forster S.
        • Housley G.
        • McKeever T.M.
        • Shaw D.E.
        Investigating the discriminative value of Early Warning Scores in patients with respiratory disease using a retrospective cohort analysis of admissions to Nottingham University Hospitals Trust over a 2-year period.
        BMJ Open. 2018; 8e020269https://doi.org/10.1136/bmjopen-2017-020269
        • Hoikka M.
        • Silfvast T.
        • TI Ala-Kokko
        Does the prehospital National Early Warning Score predict the short-term mortality of unselected emergency patients?.
        Scand J Trauma Resusc Emerg Med. 2018; 26: 48https://doi.org/10.1186/s13049-018-0514-1
        • Vihonen H.
        • Lääperi M.
        • Kuisma M.
        • Pirneskoski J.
        • Nurmi J.
        Glucose as an additional parameter to National Early Warning Score (NEWS) in prehospital setting enhances identification of patients at risk of death: an observational cohort study.
        Emerg Med J. 2020; 37: 286-292
        • Chen L.
        • Deng L.
        • Zhao H.
        • et al.
        Comparison of national early warning score, rapid emergency medicine score and acute physiology and chronic health evaluation II score for predicting outcome among emergency severe patients.
        Zhonghua Wei Zhong Bing Ji Jiu Yi Xue. 2017; 29: 1092-1096
        • Engebretsen S.
        • Bogstrand S.T.
        • Jacobsen D.
        • Vitelli V.
        • Rimstad R.
        NEWS2 versus a single-parameter system to identify critically ill medical patients in the emergency department.
        Resusc Plus. 2020; 3100020https://doi.org/10.1016/j.resplu.2020
        • Fernando S.M.
        • Fox-Robichaud A.E.
        • Rochwerg B.
        • et al.
        Prognostic accuracy of the Hamilton Early Warning Score (HEWS) and the National Early Warning Score 2 (NEWS2) among hospitalized patients assessed by a rapid response team.
        Crit Care. 2019; 23: 60-63
        • Graham C.A.
        • Leung L.Y.
        • Lo R.S.L.
        • Yeung C.Y.
        • Chan S.Y.
        • Hung K.K.C.
        NEWS and qSIRS superior to qSOFA in the prediction of 30-day mortality in emergency department patients in Hong Kong.
        Ann Med. 2020; 52: 403-412
        • Cooksley T.
        • Kitlowski E.
        • Haji-Michael P.
        Effectiveness of Modified Early Warning Score in predicting outcomes in oncology patients.
        QJM. 2012; 105: 1083-1088
        • Duvignaud A.
        • Jaspard M.
        • Etafo I.C.
        • et al.
        LASCOPE study group. Lassa fever outcomes and prognostic factors in Nigeria (LASCOPE): a prospective cohort study.
        Lancet Glob Health. 2021; 9: e469-e478https://doi.org/10.1016/S2214-109X(20)30518-0
        • Elhidsi M.
        • Rasmin M.
        • Prasenohadi
        In-hospital mortality of pulmonary tuberculosis with acute respiratory failure and related clinical risk factors.
        J Clin Tuberc Other Mycobact Dis. 2021; 23100236https://doi.org/10.1016/j.jctube.2021.100236
        • Liljehult J.
        • Christensen T.
        Early warning score predicts acute mortality in stroke patients.
        Acta Neurol Scand. 2016; 133: 261-267
        • Wang J.L.
        • Lu X.Y.
        • Xu X.H.
        • et al.
        Predictive role of monocyte-to-lymphocyte ratio in patients with Klebsiella pneumonia infection: a single-center experience.
        Medicine (Baltimore). 2019; 98: e17215https://doi.org/10.1097/MD.0000000000017215

      Linked Article

      • Prediction tools in clinical practice: Carefully read instructions before use
        European Journal of Internal MedicineVol. 98
        • Preview
          Decisions in clinical practice, especially in the acute setting where identification of potentially unstable patients is critical, are hard to be made. They rely, or should rely, on the clinical information of the patient, the best available knowledge, the expertise of the operator and the patient's preferences. Clinical prediction tools are often used as a “short-cut” to simplify and standardize such choices, without making errors.
        • Full-Text
        • PDF