Abstract
Develop and compare separate prediction models for ICU and non-ICU care for hospitalized children in four future time periods (6-12, 12-18, 18-24, and 24-30 hr) and assess these models in an independent cohort and simulated children's hospital. DESIGN: Predictive modeling used cohorts from the Health Facts database (Cerner Corporation, Kansas City, MO). SETTING: Children hospitalized in ICUs. PATIENTS: Children with greater than or equal to one ICU admission (n = 20,014) and randomly selected routine care children without ICU admission (n = 20,130) from 2009 to 2016 were used for model development and validation. An independent 2017-2018 cohort consisted of 80,089 children. INTERVENTIONS: None. MEASUREMENT AND MAIN RESULTS: Initially, we undersampled non-ICU patients for development and comparison of the models. We randomly assigned 64% of patients for training, 8% for validation, and 28% for testing in both clinical groups. Two additional validation cohorts were tested: a simulated children's hospitals and the 2017-2018 cohort. The main outcome was ICU care or non-ICU care in four future time periods based on physiology, therapy, and care intensity. Four independent, sequential, and fully connected neural networks were calibrated to risk of ICU care at each time period. Performance for all models in the test sample were comparable including sensitivity greater than or equal to 0.727, specificity greater than or equal to 0.885, accuracy greater than 0.850, area under the receiver operating characteristic curves greater than or equal to 0.917, and all had excellent calibration (all R(2) s > 0.98). Model performance in the 2017-2018 cohort was sensitivity greater than or equal to 0.545, specificity greater than or equal to 0.972, accuracy greater than or equal to 0.921, area under the receiver operating characteristic curves greater than or equal to 0.946, and R(2) s greater than or equal to 0.979. Performance metrics were comparable for the simulated children's hospital and for hospitals stratified by teaching status, bed numbers, and geographic location. CONCLUSIONS: Machine learning models using physiology, therapy, and care intensity predicting future care needs had promising performance metrics. Notably, performance metrics were similar as the prediction time periods increased from 6-12 hours to 24-30 hours.