Abstract
Clinical notes are crucial for patient assessment in the ICU but can be challenging to accurately and objectively analyze in time-constrained situations. We developed the GRACE-ICU model which integrates clinical notes and structured data to rapidly assess critical illness severity in older adults. Based on a cohort from a large U.S. teaching hospital, we fine-tuned a Clinical-Longformer model on pre-ICU notes and combined it with 10 significant structured variables via logistic regression. The receiver operating characteristic curve, calibration curve, decision curve analysis, and 11 metrics were obtained to evaluate its performance in internal, temporal, and external validations when compared with four types of baseline models. Our model outperformed the single-modal models and clinical commonly-used illness scores in both internal and temporal validation for early prediction of hospital mortality and provides interpretable, data-driven recommendations for clinical decision-making, with potential for broader applications. Further prospective studies are needed before clinical use.