Abstract
PURPOSE: To determine how to optimize the delivery of machine learning techniques in a clinical setting to detect intracranial hemorrhage (ICH) on non-contrast-enhanced CT images to radiologists to improve workflow. MATERIALS AND METHODS: In this study, a commercially available machine learning algorithm that flags abnormal noncontrast CT examinations for ICH was implemented in a busy academic neuroradiology practice between September 2017 and March 2019. The algorithm was introduced in three phases: (a) as a "pop-up" widget on ancillary monitors, (b) as a marked examination in reading worklists, and (c) as a marked examination for reprioritization based on the presence of the flag. A statistical approach, which was based on a queuing theory, was implemented to assess the impact of each intervention on queue-adjusted wait and turnaround time compared with historical controls. RESULTS: Notification with a widget or flagging the examination had no effect on queue-adjusted image wait (P > .99) or turnaround time (P = .6). However, a reduction in queue-adjusted wait time was observed between negative (15.45 minutes; 95% CI: 15.07, 15.38) and positive (12.02 minutes; 95% CI: 11.06, 12.97; P < .0001) artificial intelligence-detected ICH examinations with reprioritization. Reduced wait time was present for all order classes but was greatest for examinations ordered as routine for both inpatients and outpatients because of their low priority. CONCLUSION: The approach used to present flags from artificial intelligence and machine learning algorithms to the radiologist can reduce image wait time and turnaround times.© RSNA, 2021See also the commentary by O'Connor and Bhalla in this issue.