Abstract
BACKGROUND: Artificial intelligence (AI) has emerged as a promising tool for decision support in managing acute respiratory failure, yet its real-world clinical impact remains unclear. This scoping review identifies clinically validated AI-driven tools in this domain, focusing on the reporting of key evaluation quality measures that are a prerequisite for broader deployment. ELIGIBILITY CRITERIA: Studies were included if they compared a clinical, human factors, or health systems-related outcome of an AI-driven intervention to a control group in adult patients with acute respiratory failure. Studies were excluded if they lacked a machine learning model, compared models trained on the same dataset, assessed only model performance, or evaluated models in simulated settings. A systematic literature search was conducted in PubMed, CINAHL, and EmBase, from inception until January 2025. Each abstract was independently screened by two reviewers. One reviewer extracted data and performed quality assessment, following the DECIDE-AI framework for early-stage clinical evaluation of AI-based decision support systems. RESULTS: Of 5,987 citations, six studies met eligibility. The studies, conducted between 2012 and 2024 in Taiwan, Italy, and the U.S., included 40-2,536 patients. Four studies (67%) focused on predicting weaning from mechanical ventilation. Three (50%) of the studies demonstrated a statistically significant and clinically meaningful outcome. Studies met a median of 3.5 (IQR: 2.25-6.25) of the 17 DECIDE-AI criteria. None reported AI-related errors, malfunctions, or algorithmic fairness considerations. Only one study (17%) described user characteristics and adherence, while two (33%) assessed human-computer agreement and usability. CONCLUSIONS: Our review identified six studies evaluating AI-driven decision support tools for acute respiratory failure, with most focusing on predicting weaning from mechanical ventilation. However, methodological rigor for early clinical evaluation was inconsistent, with studies meeting few of the DECIDE-AI criteria. Notably, critical aspects such as error reporting, algorithmic fairness, and user adherence were largely unaddressed. Further high-quality assessments of reliability, usability, and real-world implementation are essential to realize the potential of these tools to transform patient care.