Abstract
INTRODUCTION: The aim of this study is to use observational methods to evaluate reliability of evidence generated by a study of the effect of glucagon-like peptide 1 receptor agonists (GLP-1RA) on chronic lower respiratory disease (CLRD) outcomes among Type-2 diabetes mellitus (T2DM) patients. RESEARCH DESIGN AND METHODS: We independently reproduced a study comparing effects of GLP-1RA versus dipeptidyl peptidase-4 inhibitors (DPP4-i) on CLRD outcomes among patients with T2DM and prior CLRD. We reproduced inputs and outputs using the original study data (national administrative claims) and evaluated the robustness of results in comparison to alternate design/analysis decisions. To evaluate generalizability, we applied an analysis protocol and conducted a meta-analysis across a research network that includes a diverse array of populations and data sources. We also produced additional analyses evaluating individual drugs within the GLP-1RA class and CLRD outcomes. RESULTS: We confirmed alignment of study inputs and outputs and closely reproduced effect estimates and sensitivity analyses. Adjusted effect estimates were robust to empirical calibration. Network meta-analysis confirmed original findings but indicated weaker effects than originally published. Meta-analysis of drugs within the GLP-1RA class against DPP4-i provided evidence that effects vary within the GLP-1RA class, indicating stronger effects for exenatide and weaker effects of dulaglutide. CONCLUSIONS: This study supports and establishes the reliability of the original study by (1) producing consistent findings in a range of alternate databases and populations, (2) demonstrating effects for multiple drugs within the GLP-1RA class, and (3) independently confirming the reproducibility of the original study and its findings. This reliability evaluation provides the beginnings of a broader framework for using standardized tools and distributed data networks to systematically interrogate the reliability of findings generated using observational data.