Abstract
The adoption of artificial intelligence (AI) in medical imaging requires careful evaluation of machine-learning algorithms. We propose the use of a "deep virtual clinical trial" (DeepVCT) method to effectively evaluate the performance of AI algorithms. In this paper, DeepVCTs have been proposed to elucidate limitations of AI applications and predictions of clinical outcomes, avoiding biases in study designs. The DeepVCT method was used to evaluate the performance of nnU-Net models in assessing volumetric breast density (VBD) from digital breast tomosynthesis (DBT) images. In total, 2,010 anatomical breast models were simulated. Projections were simulated using the acquisition geometry of a clinical DBT system. The projections were reconstructed using 0.1, 0.2, and 0.5 mm plane spacing. nnU-Net models were developed using the center-most planes of the reconstructions with the respective ground-truth. The results show that the accuracy of the nnU-Net improves significantly with DBT images reconstructed with 0.1 mm plane spacing (78.4×205.3×40.1 mm(3)). The segmentations resulted in Dice values up to 0.84 with area under the receiver operating characteristic curve of 0.92. The optimization of plane spacing for VBD assessment was used as an exemplar of a DeepVCT application, allowing us to interpret better the input parameters and outcomes of the nnU-Net. Thus, DeepVCTs can provide a plethora of evidence to predict the efficacy of these algorithms using large-scale simulation-based data.