External Validation of the Prognostic Prediction Model for 4-Year Risk of Metabolic Syndrome in Adults: A Retrospective Cohort Study

Purpose: A prediction model for 4-year risk of metabolic syndrome in adults was previously developed and internally validated. However, external validity or generalizability for this model was not assessed so it is not appropriate for clinical application. We aimed to externally validate this model based on a retrospective cohort. Patients and Methods: A retrospective cohort design and a temporal validation strategy were used in this study based on a dataset from 1 January 2015 to 31 December 2018. Multiple imputation was used for missing values. Model performance was evaluated by using discrimination, calibration (calibration plot, calibration slope, and calibration intercept), overall performance (Brier score), and decision curve analysis. Results: In external validation, the C-statistic was 0.782 (95% CI, 0.771-0.793). The calibration plot shows good calibration, calibration slope was 1.006 (95% CI, -0.011- 1.063), and calibration intercept was -0.045 (95% CI, -0.113-0.022). Brier score was 0.164.The discrimination and calibration of the prediction model were good in temporal external validation. Conclusion: The discrimination and calibration of the prediction model were satisfactory in the temporal external validation. However, clinicians should be aware that this prediction model was developed and validated in a tertiary setting. It is strongly recommended that further studies validate this model in international cohorts and large, prospective cohorts in different institutions.