Abstract
Machine learning (ML) models for predicting 72-hour unscheduled return visits (URVs) for patients with abdominal pain in the emergency department (ED) were developed in a previous study. This study refined the data to adjust previous prediction models and evaluated the model performance in future data validation during the COVID-19 era. We aimed to evaluate the practicality of the ML models and compare the URVs before and during the COVID-19 pandemic. We used electronic health records from Chang Gung Memorial Hospital from 2018 to 2019 as a training dataset, and various machine learning models, including logistic regression (LR), random forest (RF), extreme gradient boosting (XGB), and voting classifier (VC) were developed and subsequently used to validate against the 2020 to 2021 data. The models highlighted several determinants for 72-hour URVs, including patient age, prior ER visits, specific vital signs, and medical interventions. The LR, XGB, and VC models exhibited the same AUC of 0.71 in the testing set, whereas the VC model displayed a higher F1 score (0.21). The XGB model demonstrated the highest specificity (0.99) and precision (0.64) but the lowest sensitivity (0.01). Among these models, the VC model showed the most favorable, balanced, and comprehensive performance. Despite the promising results, the study illuminated challenges in predictive modeling, such as the unforeseen influences of global events, such as the COVID-19 pandemic. These findings not only highlight the significant potential of machine learning in augmenting emergency care but also underline the importance of iterative refinement in response to changing real-world conditions.