Abstract
PURPOSE: There exists a barrier between developing and disseminating risk prediction models in clinical settings. We hypothesize that this barrier may be lifted by demonstrating the utility of these models using incomplete data that are collected in real clinical sessions, as compared with the commonly used research cohorts that are meticulously collected. MATERIALS AND METHODS: Genetic counselors (GCs) collect family history when patients (ie, probands) come to MD Anderson Cancer Center for risk assessment of Li-Fraumeni syndrome, a genetic disorder characterized by deleterious germline mutations in the TP53 gene. Our clinical counseling-based (CCB) cohort consists of 3,297 individuals across 124 families (522 cases of single primary cancer and 125 cases of multiple primary cancers). We applied our software suite LFSPRO to make risk predictions and assessed performance in discrimination using AUC and in calibration using observed/expected (O/E) ratio. RESULTS: For prediction of deleterious TP53 mutations, we achieved an AUC of 0.78 (95% CI, 0.71 to 0.85) and an O/E ratio of 1.66 (95% CI, 1.53 to 1.80). Using the LFSPRO.MPC model to predict the onset of the second cancer, we obtained an AUC of 0.70 (95% CI, 0.58 to 0.82). Using the LFSPRO.CS model to predict the onset of different cancer types as the first primary, we achieved AUCs between 0.70 and 0.83 for sarcoma, breast cancer, or other cancers combined. CONCLUSION: We describe a study that fills in the critical gap in knowledge for the utility of risk prediction models. Using a CCB cohort, our previously validated models have demonstrated good performance and outperformed the standard clinical criteria. Our study suggests that better risk counseling may be achieved by GCs using these already-developed mathematical models.