Abstract
Exam protocoling is a significant non-interpretive task burden for radiologists. The purpose of this work was to develop a natural language processing (NLP) artificial intelligence (AI) solution for automated protocoling of standard abdomen and pelvic magnetic resonance imaging (MRI) exams from basic associated order information and patient metadata. This Institutional Review Board exempt retrospective study used de-identified metadata from consecutive adult abdominal and pelvic MRI scans performed at our institution spanning 2.5 years from 2019 to 2021 to fine-tune an AI model to predict the exam protocol. The NLP algorithm Bidirectional Encoder Representations from Transformers (BERT) was employed in sequence classification mode. Twelve months of data from the COVID pandemic were excluded to avoid bias from known practice and referral pattern disruptions, with approximately 46,000 MRI exams in the resulting cohort. The final trained model had an accuracy of 88.5% with a Matthews correlation coefficient of 0.874, a true positive rate of 0.872, and a true negative rate of 0.995. Subsequent expert review of the errors performed to satisfy departmental leadership showed 81.9% were in fact correct or reasonable alternative protocols, yielding real-world performance accuracy of 97.9%. We conclude that NLP algorithms, including "smaller" large language models like the BERT family often overlooked today, can predict MRI imaging protocols for the abdomen and pelvis with high real-world performance, offering to decrease radiologists' non-interpretive task load and increasing departmental efficiency.