Abstract
BACKGROUND: Diabetes management is complex, and program personalization has been identified to enhance engagement and clinical outcomes in diabetes management programs. However, 50% of individuals living with diabetes are unable to achieve glycemic control, presenting a gap in the delivery of self-management education and behavior change. Machine learning and recommender systems, which have been used within the health care setting, could be a feasible application for diabetes management programs to provide a personalized user experience and improve user engagement and outcomes. OBJECTIVE: This study aims to evaluate machine learning models using member-level engagements to predict improvement in estimated A(1c) and develop personalized action recommendations within a remote diabetes monitoring program to improve clinical outcomes. METHODS: A retrospective study of Livongo for Diabetes member engagement data was analyzed within five action categories (interacting with a coach, reading education content, self-monitoring blood glucose level, tracking physical activity, and monitoring nutrition) to build a member-level model to predict if a specific type and level of engagement could lead to improved estimated A(1c) for members with type 2 diabetes. Engagement and improvement in estimated A(1c) can be correlated; therefore, the doubly robust learning method was used to model the heterogeneous treatment effect of action engagement on improvements in estimated A(1c). RESULTS: The treatment effect was successfully computed within the five action categories on estimated A(1c) reduction for each member. Results show interaction with coaches and self-monitoring blood glucose levels were the actions that resulted in the highest average decrease in estimated A(1c) (1.7% and 1.4%, respectively) and were the most recommended actions for 54% of the population. However, these were found to not be the optimal interventions for all members; 46% of members were predicted to have better outcomes with one of the other three interventions. Members who engaged with their recommended actions had on average a 0.8% larger reduction in estimated A(1c) than those who did not engage in recommended actions within the first 3 months of the program. CONCLUSIONS: Personalized action recommendations using heterogeneous treatment effects to compute the impact of member actions can reduce estimated A(1c) and be a valuable tool for diabetes management programs in encouraging members toward actions to improve clinical outcomes.