Abstract
BACKGROUND: Public health authorities have been recommending interventions such as physical distancing and face masks, to curtail the transmission of coronavirus disease (COVID-19) within the community. Public perceptions toward such interventions should be identified to enable public health authorities to effectively address valid concerns. The Health Belief Model (HBM) has been used to characterize user-generated content from social media during previous outbreaks, with the aim of understanding the health behaviors of the public. OBJECTIVE: This study is aimed at developing and evaluating deep learning-based text classification models for classifying social media content posted during the COVID-19 outbreak, using the four key constructs of the HBM. We will specifically focus on content related to the physical distancing interventions put forth by public health authorities. We intend to test the model with a real-world case study. METHODS: The data set for this study was prepared by analyzing Facebook comments that were posted by the public in response to the COVID-19-related posts of three public health authorities: the Ministry of Health of Singapore (MOH), the Centers for Disease Control and Prevention, and Public Health England. The comments made in the context of physical distancing were manually classified with a Yes/No flag for each of the four HBM constructs: perceived severity, perceived susceptibility, perceived barriers, and perceived benefits. Using a curated data set of 16,752 comments, gated recurrent unit-based recurrent neural network models were trained and validated for text classification. Accuracy and binary cross-entropy loss were used to evaluate the model. Specificity, sensitivity, and balanced accuracy were used to evaluate the classification results in the MOH case study. RESULTS: The HBM text classification models achieved mean accuracy rates of 0.92, 0.95, 0.91, and 0.94 for the constructs of perceived susceptibility, perceived severity, perceived benefits, and perceived barriers, respectively. In the case study with MOH Facebook comments, specificity was above 96% for all HBM constructs. Sensitivity was 94.3% and 90.9% for perceived severity and perceived benefits, respectively. In addition, sensitivity was 79.6% and 81.5% for perceived susceptibility and perceived barriers, respectively. The classification models were able to accurately predict trends in the prevalence of the constructs for the time period examined in the case study. CONCLUSIONS: The deep learning-based text classifiers developed in this study help to determine public perceptions toward physical distancing, using the four key constructs of HBM. Health officials can make use of the classification model to characterize the health behaviors of the public through the lens of social media. In future studies, we intend to extend the model to study public perceptions of other important interventions by public health authorities.