External validation of the CACE-HF risk score for mortality in patients with heart failure


      • To improve our management of HF we need tools to optimize the available resources.
      • The prognostic models help us differentiate the life expectancy of the HF patients.
      • We have externally validated a risk model to predict 1-year mortality.
      • The model was developed and validated in real-world cohorts of HF patients.



      To validate externally the CACE-HF clinical prediction rule, which predicts 1-year mortality in patients with heart failure (HF).


      We performed an external validation of the CACE-HF risk score in patients included in the RICA heart failure registry who had completed 1 year of follow-up, comparing the characteristics of the derivation and validation cohorts. The performance of the risk score was evaluated in terms of calibration, using calibration-in-the-large (a), calibration slope (b), and the Hosmer-Lemeshow test, and in terms of discrimination, using the area under the ROC curve.


      In total, 3337 patients were included in the validation cohort. There were no significant differences between the derivation and validation cohorts in 1-year mortality (24.63% vs. 22.98%) or in the risk score and risk classes. The discrimination capacity in the validation cohort was slightly lower, 0.67 (95% CI: 0.65, 0.69), compared to that of the derivation cohort. Calibration results were a −0.05 (95% CI: −0.14, 0.03), indicating that the average predictions did not differ from the average outcome frequency, and b = 0.75 (95% CI: 0.64, 0.86), indicating a modest inconsistency in predictor effects. Observed mortality versus predicted mortality according to the deciles and risk classes were very similar in both cases, indicating good calibration.


      The results of the external validation of the CACE-HF risk score show that although the capacity for discrimination was slightly lower than in the derivation cohort, the calibration was excellent. This tool, therefore, can assist in decision-making in the management of these patients.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to European Journal of Internal Medicine
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Ponikowski P.
        • Voors A.A.
        • Anker S.D.
        • et al.
        ESC Guidelines for the diagnosis and treatment of acute and chronic heart failure: the Task Force for the diagnosis and treatment of acute and chronic heart failure of the European Society of Cardiology (ESC). Developed with the special contribution of the Heart Failure Association (HFA) of the ESC.
        Eur J Heart Fail. 2016; 2016: 891-975
        • Ouwerkerk W.
        • Voors A.A.
        • Zwinderman A.H.
        Factors influencing the predictive power of models for predicting mortality and/or heart failure hospitalization in patients with heart failure.
        JACC Heart Fail. 2014; 2: 429-436
        • Altman D.G.
        • Vergouwe Y.
        • Royston P.
        • et al.
        Prognosis and prognostic research: validating a prognostic model.
        BMJ. 2009; 338: b605
        • Escobar A.
        • Garcia-Perez L.
        • Navarro G.
        • et al.
        A one-year mortality clinical prediction rule for patients with heart failure.
        Eur J Intern Med. 2017; 44: 49-54
        • Trullas J.C.
        • Miro O.
        • Formiga F.
        • et al.
        The utility of heart failure registries: a descriptive and comparative study of two heart failure registries.
        Postgrad Med J. 2016; 92: 260-266
        • Trullas J.C.
        • Formiga F.
        • Montero M.
        • et al.
        Paradox of obesity in heart failure: results from the Spanish RICA Registry.
        Med Clin (Barc). 2011; 137: 671-677
        • Debray T.P.
        • Vergouwe Y.
        • Koffijberg H.
        • et al.
        A new framework to enhance the interpretation of external validation studies of clinical prediction models.
        J Clin Epidemiol. 2015; 68: 279-289
        • Hanley J.A.
        • McNeil B.J.
        The meaning and use of the area under a receiver operating characteristic (ROC) curve.
        Radiology. 1982; 143: 29-36
        • Hosmer D.W.
        • Lemeshow S.
        Applied logistic regression.
        Wiley, New York1989
        • Steyerberg E.W.
        • Vickers A.J.
        • Cook N.R.
        • et al.
        Assessing the performance of prediction models: a framework for traditional and novel measures.
        Epidemiology. 2010; 21: 128-138
        • Janssen K.J.
        • Moons K.G.
        • Kalkman C.J.
        • et al.
        Updating methods improved the performance of a clinical prediction model in new patients.
        J Clin Epidemiol. 2008; 61: 76-86
        • Steyerberg E.W.
        Clinical prediction models. A practical approach to development, validation, and updating.
        Springer, 2009
        • Moons K.G.
        • Kengne A.P.
        • Grobbee D.E.
        • et al.
        Risk prediction models: II. External validation, model updating, and impact assessment.
        Heart. 2012; 98: 691-698
        • Fonarow G.C.
        Clinical risk prediction tools in patients hospitalized with heart failure.
        Rev Cardiovasc Med. 2012; 13: 14-23
        • Rahimi K.
        • Bennett D.
        • Conrad N.
        • et al.
        Risk prediction in patients with heart failure: a systematic review and analysis.
        JACC Heart Fail. 2014; 2: 440-446
        • Collins G.S.
        • de Groot J.A.
        • Dutton S.
        • et al.
        External validation of multivariable prediction models: a systematic review of methodological conduct and reporting.
        BMC Med Res Methodol. 2014; 14: 40