Abstract
BACKGROUND: Acute pancreatitis (AP) represents a critical medical condition where timely and precise prediction of in-hospital mortality is crucial for guiding optimal clinical management. This study focuses on the development of advanced machine learning (ML) models to accurately predict in-hospital mortality among AP patients admitted to intensive care unit (ICU). METHOD: Our study utilized data from three distinct sources: the Medical Information Mart for Intensive Care III (MIMIC-III), MIMIC-IV databases, and Beijing Chaoyang Hospital. We systematically developed and evaluated 11 distinct machine learning (ML) models, employing a comprehensive set of evaluation metrics to assess model performance, including the area under the curve (AUC). To enhance interpretability and identify key predictive features, we implemented Shapley Additive Explanations (SHAP) analysis for the top-performing model. Furthermore, we developed a streamlined version of the model through strategic feature reduction, followed by rigorous hyperparameter optimization (HPO) to maximize predictive performance. To facilitate clinical implementation, we designed and deployed an intuitive web-based calculator, enabling convenient access and practical application of our optimized predictive model. RESULT: The study analyzed 1802 AP patients, with 266 (14.8%) experiencing in-hospital mortality. A set of 27 features was utilized to construct various models, and among them, CatBoost demonstrated the highest performance in both the validation and test sets. To create a more concise model, we selected the top 13 features. After HPO, the AUC in the test set reached 0.835 (95% CI: 0.793-0.872), the AUC in the external validation from Beijing Chaoyang hospital was 0.782 (95% CI: 0.699-0.860). CONCLUSION: ML models have shown promising reliability in predicting in-hospital mortality among patients with AP in the ICU. Among these models, the CatBoost model exhibits superior predictive performance, providing valuable assistance to clinical practitioners in identifying high-risk patients and facilitating early interventions to enhance prognosis. The development of a compact model and a web-based calculator further enhances the convenience of using these models in clinical practice.