Abstract
INTRODUCTION AND OBJECTIVES: An AI model that performs well during training does not guarantee similar performance in clinical practice and should be carefully evaluated before implementation. We aimed to evaluate a voxel-level trained AI model (AUROC 0.87), which utilizes a three-dimensional multiparametric transrectal prostate ultrasound (3D mpUS) to identify clinically significant prostate cancer (csPCa). MATERIALS AND METHODS: We included patients with csPCa (Grade Group ≥ 2 and scheduled for radical prostatectomy (RP)) and without csPCa (PI-RADS ≤ 2 and/or negative systematic biopsies). Histopathology of RP specimens provided the csPCa reference standard. 3D mpUS consisted of grayscale, contrast-enhanced ultrasound, and shear-wave elastography using automated acquisition. We assessed patient-level diagnostic accuracy by comparing the results of simulated targeted biopsies based on the AI model with the reference standard in internal and external evaluation. Patients without csPCa and RP reference standard were used to determine specificity. RESULTS: Based on internal evaluation of 250 patients, a sensitivity of 0.82 (CI 0.75 to 0.87) and specificity of 0.43 (CI 0.32 to 0.55) was reached for ISUP ≥ 2. For ISUP ≥ 3, this was 0.90 (CI 0.83-0.95) and 0.39 (CI 0.31-0.47). In the external evaluation of 77 patients, the sensitivity for ISUP ≥ 2 was 0.81 (CI 0.65-0.90), with a specificity of 0.42 (CI 0.28-0.57). For ISUP ≥ 3, this was 0.96 (CI 0.78-0.99) and 0.42 (CI 0.30-0.55). CONCLUSIONS: The AI model based on 3D mpUS showed consistent patient-level performance for csPCa detection in internal and external evaluation, comparable to voxel-level analysis. These suggest strong generalizability and support prospective clinical trials. TRIAL REGISTRATION: NCT04605276. KEY POINTS: Question Does the diagnostic performance of a 3D multiparametric ultrasound-based AI model translate from voxel-level training to patient-level biopsy simulation? Findings Simulated biopsy performance aligned with voxel-level results, showing robust csPCa detection and supporting the model's generalizability across independent datasets. Clinical relevance The AI model's consistent biopsy simulation performance confirms its readiness for clinical evaluation and suggests diagnostic value in MRI-constrained settings.