Abstract
In a recent 12-week smoking cessation trial, varenicline tartrate failed to show significant improvements in enhancing end-of-treatment abstinence when compared with placebo among adolescents and young adults. The original analysis aimed to assess the average effect across the entire population using timeline followback methods, which typically involve overdispersed binomial counts. We instead propose to investigate treatment effect heterogeneity among latent classes of participants using a Bayesian beta-binomial piecewise linear growth mixture model specifically designed to address longitudinal overdispersed binomial responses. Within each class, we fit a piecewise linear beta-binomial mixed model with random changepoints for each study group to detect critical windows of treatment efficacy. Using this model, we can cluster subjects who share similar characteristics, estimate the class-specific mean abstinence trends for each study group, and quantify the treatment effect over time within each class. Our analysis identified two classes of subjects: one comprising high-abstinent individuals, typically young adults and light smokers, in which varenicline led to improved abstinence; and another comprising low-abstinent individuals for whom varenicline showed no discernible effect. These findings highlight the importance of tailoring varenicline to specific participant subgroups, thereby advancing precision medicine in smoking cessation studies.