Abstract
Patient safety and high treatment quality are essential in modern healthcare, but analyzing safety incidents for insights is labor-intensive and inconsistent process. To address this, we developed the Artificial Intelligence-based Incident Analysis and Learning System (AI-ILS), trained on 1548 expertly curated incidents categorized by the Human Factors Analysis and Classification System (HFACS). AI-ILS identifies latent safety threats and classifies incident causes with high accuracy, achieving an average AUROC of 0.92, MCC of 0.72, and overall accuracy of 79%. In testing on 350 real-world clinical incidents, AI-ILS showed 88% concordance with expert reviewers and operated 29 times faster than manual analysis. We deployed and validated AI-ILS using real-world radiation oncology data, where it improved retrospective incident analysis at our institution by generating aggregated HFACS-based results and addressing challenges related to inconsistent review processes and lack of standardized taxonomies.