Abstract
BACKGROUND: Assessing the generalizability of randomized clinical trials (RCTs) to real-world patients remains challenging. We propose a multidimensional metric to quantify the representativeness of an RCT cohort in an electronic health record (EHR) population and estimate real-world effects based on individualized treatment effects observed in the RCT. METHODS: We identified 65 clinical prerandomization characteristics of patients with heart failure with preserved ejection fraction within the TOPCAT (Treatment of Preserved Cardiac Function Heart Failure with an Aldosterone Antagonist Trial) and extracted those features in similar patients in EHR data from 4 hospitals in the Yale New Haven Health System. We then assessed the real-world generalizability of TOPCAT by developing a novel statistic, the phenotypic distance metric, to quantify the representation of TOPCAT participants within EHR patients. Finally, applying a machine learning method to learn individualized treatment effect in TOPCAT participants stratified by region, the United States (US) and Eastern Europe (EE), we predicted spironolactone benefit within the EHR cohorts. RESULTS: There were 3445 patients in TOPCAT (median age 69, interquartile range [IQR], 61-76 years, 52% women) and 8121 patients with heart failure with preserved ejection fraction across 4 hospitals (median age range 77, IQR, 68-86; years to 85; IQR, 77-91 years, 54% to 62% women). Across covariates, the EHR patients were more similar to each other than the TOPCAT-US participants (median standardized mean difference 0.065, IQR, 0.011-0.144 versus median standardized mean difference 0.186, IQR, 0.040-0.479). The phenotypic distance metric found a higher generalizability of the TOPCAT-US participants to the EHR patients than the TOPCAT-EE participants. Using a TOPCAT-US-derived model of individualized treatment effect, all EHR patients were predicted to derive benefit from spironolactone treatment, while a TOPCAT-EE-derived model predicted 13% of EHR patients to derive benefit. CONCLUSIONS: This novel multidimensional metric evaluates the real-world representativeness of RCT participants against corresponding patients in the EHR, enabling the evaluation of an RCT's implication for real-world patients.