Abstract
BACKGROUND: Digital health interventions are increasingly promoted in healthcare and prevention practices due to their potential for reaching key populations in a cost-efficient manner. Yet there has been limited research on how to effectively implement them with pragmatic approaches that can facilitate scale-up. Keep It Up! (KIU!) 3.0 was a hybrid type 3 implementation-effectiveness trial comparing two delivery strategies (i.e. trial arms) of an HIV prevention intervention for cisgender, young men who have sex with men. We aimed to determine the level of pragmatism of our two-armed trial before and after changes to the county-randomized design. METHODS: We applied different versions of the PRagmatic Explanatory Continuum Indicator Summary (PRECIS) tool to the two trial arms: delivery of KIU! by community-based organizations (CBO) versus centralized, direct-to-consumer (DTC) delivery. We scored PRECIS-2 for the original study design and the modified design in which the DTC strategy expanded nationally. We applied PRECIS-2-PS to the modified study design. Nine coders in three groups independently scored the tools. Scores were iteratively discussed to arrive at one consensus score per domain, tool, design stage, and arm. We plotted results using the PRECIS-2 and PRECIS-2-PS wheels and averaged domains scores to describe overall score along the Pragmatic-Explanatory Continuum. RESULTS: Using PRECIS-2, the trial was on the pragmatic side of the spectrum for both arms and design stages, with average ratings ranging from 3.89-4.33. Both arms were highly pragmatic in the original and modified design in the Setting and Primary Analysis domains and least pragmatic in the Follow-up domain. In the modified trial design, the CBO and DTC arms again scored rather pragmatic using the PRECIS-2-PS tool, but CBO arm scored higher in the eligibility, recruitment, and organization domains compared to PRECIS-2 (5 vs. 4, respectively). CONCLUSIONS: Application of both the PRECIS-2 and PRECIS-2-PS tools validated the pragmatic design of KIU! 3.0 as originally designed and after modifications during trial implementation. Our findings highlight instances where one tool may be more suitable than the other to assess the pragmatic-explanatory continuum for emerging digital health interventions delivered in diverse settings and with different implementation strategies.