Abstract
PURPOSE: To develop and validate a deep learning (DL) algorithm to identify poor-quality lateral airway radiographs. MATERIALS AND METHODS: A total of 1200 lateral airway radiographs obtained in emergency department patients between January 1, 2000, and July 1, 2019, were retrospectively queried from the picture archiving and communication system. Two radiologists classified each radiograph as adequate or inadequate. Disagreements were adjudicated by a third radiologist. The radiographs were used to train and test the DL classifiers. Three technologists and three different radiologists classified the images in the test dataset, and their performance was compared with that of the DL classifiers. RESULTS: The training set had 961 radiographs and the test set had 239. The best DL classifier (ResNet-50) achieved sensitivity, specificity, and area under the receiver operating characteristic curve of 0.90 (95% confidence interval [CI]: 0.86, 0.94), 0.82 (95% CI: 0.76, 0.90), and 0.86 (95% CI: 0.81, 0.91), respectively. Interrater agreement for technologists was fair (Fleiss κ, 0.36 [95% CI: 0.29, 0.43]), while that for radiologists was moderate (Fleiss κ, 0.59 [95% CI: 0.52, 0.66]). Cohen κ value comparing the consensus rating of ResNet-50 iterations from fivefold cross-validation, consensus technologists' rating, and consensus radiologists' rating to the ground truth were 0.76 (95% CI: 0.63, 0.89), 0.49 (95% CI: 0.37, 0.61), and 0.66 (95% CI: 0.54, 0.78), respectively. CONCLUSION: The development and validation of DL classifiers to distinguish between adequate and inadequate lateral airway radiographs is reported. The classifiers performed significantly better than a group of technologists and as well as the radiologists.© RSNA, 2020.