Abstract
Delirium is an acute syndrome characterized by fluctuating attention, cognitive impairment, and severe disorganization of behavior, which has been shown to affect up to 31% of patients in the intensive care unit (ICU). Early detection can enable timely interventions and improved health outcomes. While artificial intelligence (AI) models have shown great potential for ICU delirium prediction using structured electronic health records (EHR), most studies have either not leveraged state-of-the-art AI models, been limited to single-center cohorts, or relied on small datasets for development and validation. In this study, we introduce DeLLiriuM, a novel LLM-based delirium prediction model that utilizes EHR data from the first 24 hours of ICU admission to estimate a patient's risk of developing delirium for the remainder of their ICU stay. We developed and validated DeLLiriuM using ICU admissions from 104,303 patients across 195 hospitals in three large databases: the eICU Collaborative Research Database, the Medical Information Mart for Intensive Care (MIMIC)-IV, and the University of Florida's Integrated Data Repository. Our DeLLiriuM model achieved superior performance compared to all baseline models on the external validation set, measured by the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC) metric. DeLLiriuM attained AUROC 82.4 (95% confidence interval 81.8-83.0) and AUPRC 11.8 (95% confidence interval 11.3-12.4) across 77,543 patients spanning 194 hospitals. Our approach of transforming structured EHR data into an unstructured text format, the primary data modality for LLMs, enables our DeLLiriuM model to capture clinical contextual information, resulting in improved predictive performance. To the best of our knowledge, DeLLiriuM is the first LLM-based delirium prediction tool for the ICU that utilizes structured EHR data with LLMs rather than clinical notes with LLMs or traditional structured feature representations used in AI models.