Abstract
BACKGROUND: Statistical prediction tools are increasingly common, but there is considerable disagreement about how they should be evaluated. Three tools--Partin tables, the European Society for Urological Oncology (ESUO) criteria, and the Gallina nomogram--have been proposed for the prediction of seminal vesicle invasion (SVI) in patients with clinically localized prostate cancer who are candidates for a radical prostatectomy. OBJECTIVES: Using different statistical methods, we aimed to determine which of these tools should be used to predict SVI. DESIGN, SETTINGS, AND PARTICIPANTS: The independent validation cohort consisted of 2584 patients treated surgically for clinically localized prostate cancer at four North American tertiary care centers between 2002 and 2007. INTERVENTIONS: Robot-assisted laparoscopic radical prostatectomy. OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS: Primary outcome was the presence of SVI. Traditional (area under the receiver operating characteristic [ROC] curve, calibration plots, the Brier score, sensitivity and specificity, positive and negative predictive value) and novel (decision curve analysis and predictiveness curves) statistical methods quantified the predictive abilities of the three models. RESULTS AND LIMITATIONS: Traditional statistical methods (ie, ROC plots and Brier scores) could not clearly determine which one of the three SVI prediction tools should be preferred. For example, ROC plots and Brier scores seemed biased against the binary decision tool (ESUO criteria) and gave discordant results for the continuous predictions of the Partin tables and the Gallina nomogram. The results of the calibration plots were discordant with those of the ROC plots. Conversely, the decision curve indicated that the Partin tables represent the best strategy for stratifying the risk of SVI, resulting in the highest net benefit within the whole range of threshold probabilities. CONCLUSIONS: When predicting SVI, surgeons should prefer the Partin tables over the ESUO criteria and the Gallina nomogram because this tool provided the highest net benefit. In contrast to traditional statistical methods, decision curve analysis gave an unambiguous result applicable to both continuous and binary models, providing an insight into clinical utility.