Abstract
OBJECTIVE: There has been increased interest in interventions to promote hepatocellular carcinoma (HCC) surveillance given low utilization and high proportions of late stage detection. Accurate prediction of patients likely versus unlikely to respond to interventions could allow a cost-effective approach to outreach and facilitate targeting more intensive interventions to likely non-responders. DESIGN: We conducted a secondary analysis of a randomized clinical trial evaluating a mailed outreach strategy to promote HCC surveillance among 1200 cirrhosis patients at a safety-net health system between December 2014 and March 2017. We developed regularized logistic regression (RLR) and gradient boosting machine (GBM) algorithm models to predict surveillance completion during each of the 3 screening rounds in a training set (n = 960). Model performance was assessed using multiple performance metrics in an independent test set (n = 240). RESULTS: Among 1200 patients, surveillance was completed in 41-47% of patients over the three rounds. The RLR and GBM models demonstrated good discriminatory accuracy, with area under receiver operating characteristic (AUROC) curves of 0.67 and 0.66 respectively in the first surveillance round and improved to 0.77 by the third surveillance round after incorporating prior screening behavior as a feature. Additional performance characteristics including the Brier score, Hosmer-Lemeshow test and reliability diagrams were also evaluated. The most important variables for the predictive model were prior screening completion status and past primary care contact. CONCLUSIONS: Predictive models can help stratify patients' likelihood to respond to surveillance outreach invitations, facilitating tailored strategies to maximize effectiveness and cost-effectiveness of HCC surveillance population health programs.