Abstract
BACKGROUND: Overcrowded emergency rooms might degrade the quality of care and overload the clinic staff. Assessing unscheduled return visits (URVs) to the emergency department (ED) is a quality assurance procedure to identify ED-discharged patients with a high likelihood of bounce-back, to ensure patient safety, and ultimately to reduce medical costs by decreasing the frequency of URVs. The field of machine learning (ML) has evolved considerably in the past decades, and many ML applications have been deployed in various contexts. OBJECTIVE: This study aims to develop an ML-assisted framework that identifies high-risk patients who may revisit the ED within 72 hours after the initial visit. Furthermore, this study evaluates different ML models, feature sets, and feature encoding methods in order to build an effective prediction model. METHODS: This study proposes an ML-assisted system that extracts the features from both structured and unstructured medical data to predict patients who are likely to revisit the ED, where the structured data includes patients' electronic health records, and the unstructured data is their medical notes (subjective, objective, assessment, and plan). A 5-year dataset consisting of 184,687 ED visits, along with 324,111 historical electronic health records and the associated medical notes, was obtained from Kaohsiung Veterans General Hospital, a tertiary medical center in Taiwan, to evaluate the proposed system. RESULTS: The evaluation results indicate that incorporating convolutional neural network-based feature extraction from unstructured ED physician narrative notes, combined with structured vital signs and demographic data, significantly enhances predictive performance. The proposed approach achieves an area under the receiver operating characteristic curve of 0.705 and a recall of 0.718, demonstrating its effectiveness in predicting URVs. These findings highlight the potential of integrating structured and unstructured clinical data to improve predictive accuracy in this context. CONCLUSIONS: The study demonstrates that an ML-assisted framework may be applied as a decision support tool to assist ED clinicians in identifying revisiting patients, although the model's performance may not be sufficient for clinic implementation. Given the improvement in the area under the receiver operating characteristic curve, the proposed framework should be further explored as a workable decision support tool to pinpoint ED patients with a high risk of revisit and provide them with appropriate and timely care.