Abstract
Emergency department (ED) crowding necessitates rapid, accurate risk stratification to optimize care and resource allocation. Traditional clinical prediction tools like NEWS, APACHE II, and SOFA score have limited generalizability or rely on extensive inputs, including vitals. We developed RISK(INDEX), a machine-learning tool predicting 31-day mortality using routine laboratory values, age, and sex. To evaluate the clinical impact of a machine learning-based risk score in routine care, we conducted an investigator-initiated, open-label, randomized, non-inferiority trial (MARS-ED) at the Maastricht University Medical Center+ ED. Adult patients ( ≥ 18 years) assessed by an internal medicine specialist and with ≥4 laboratory tests were eligible. Patients were randomized 1:1 using computer-generated permuted blocks to standard care (n = 659) or standard care plus access to the RISK(INDEX) (n = 644). No blinding was possible because physicians needed to view the RISK(INDEX). The primary outcomes for this study were the prognostic accuracy for 31-day mortality and clinical impact of the RISK(INDEX). In total, 1303 participants were analyzed. RISK(INDEX)'s prognostic accuracy matched or outperformed clinical intuition (AUROC 0.84 vs. 0.73-0.76) and was statistically higher than NEWS, APACHE II, and SOFA prediction tools (AUROC 0.65-0.75). RISK(INDEX) predictions aligned with clinicians' expectations in only about half of cases, with highest discordance among less experienced physicians. Despite its prognostic accuracy, the RISK(INDEX) did not alter treatment plans (1/644 changes; 0.16%) or clinical outcomes, and clinicians perceived low added value. No adverse events related to the intervention occurred, and recruitment was completed as planned. These findings show that prognostic accuracy alone is insufficient to achieve clinical impact in the ED and that user-centered, actionable model design is needed to ensure relevance, trust, and responsiveness. ClinicalTrials.gov registration: NCT05497830.