Abstract
BACKGROUND: Deep learning (DL)-based automatic segmentation models can expedite manual segmentation yet require resource-intensive fine-tuning before deployment on new datasets. The generalizability of DL methods to new datasets without fine-tuning is not well characterized. PURPOSE: Evaluate the generalizability of DL-based models by deploying pretrained models on independent datasets varying by MR scanner, acquisition parameters, and subject population. STUDY TYPE: Retrospective based on prospectively acquired data. POPULATION: Overall test dataset: 59 subjects (26 females); Study 1: 5 healthy subjects (zero females), Study 2: 8 healthy subjects (eight females), Study 3: 10 subjects with osteoarthritis (eight females), Study 4: 36 subjects with various knee pathology (10 females). FIELD STRENGTH/SEQUENCE: A 3-T, quantitative double-echo steady state (qDESS). ASSESSMENT: Four annotators manually segmented knee cartilage. Each reader segmented one of four qDESS datasets in the test dataset. Two DL models, one trained on qDESS data and another on Osteoarthritis Initiative (OAI)-DESS data, were assessed. Manual and automatic segmentations were compared by quantifying variations in segmentation accuracy, volume, and T2 relaxation times for superficial and deep cartilage. STATISTICAL TESTS: Dice similarity coefficient (DSC) for segmentation accuracy. Lin's concordance correlation coefficient (CCC), Wilcoxon rank-sum tests, root-mean-squared error-coefficient-of-variation to quantify manual vs. automatic T2 and volume variations. Bland-Altman plots for manual vs. automatic T2 agreement. A P value < 0.05 was considered statistically significant. RESULTS: DSCs for the qDESS-trained model, 0.79-0.93, were higher than those for the OAI-DESS-trained model, 0.59-0.79. T2 and volume CCCs for the qDESS-trained model, 0.75-0.98 and 0.47-0.95, were higher than respective CCCs for the OAI-DESS-trained model, 0.35-0.90 and 0.13-0.84. Bland-Altman 95% limits of agreement for superficial and deep cartilage T2 were lower for the qDESS-trained model, ±2.4 msec and ±4.0 msec, than the OAI-DESS-trained model, ±4.4 msec and ±5.2 msec. DATA CONCLUSION: The qDESS-trained model may generalize well to independent qDESS datasets regardless of MR scanner, acquisition parameters, and subject population. EVIDENCE LEVEL: 1 TECHNICAL EFFICACY: Stage 1.