Abstract
BACKGROUND: Online questionnaires are widely used for large-scale screening. However, careless responding (CR) from participants can compromise the reliability of screening outcomes. Prior studies have focused on the effects of individual and environmental factors on CR, but the effect of questionnaire type remains underexplored. OBJECTIVE: This study investigates the individual factors influencing CR in online mental health screening and assesses how the effect of these factors varies across different psychological questionnaires. METHODS: This study analyzed data from 24,367 participants across 4 questionnaires (PHQ-9 [Patient Health Questionnaire-9], PSS [Perceived Stress Scale], ISI [Insomnia Severity Index], and GAD-7 [Generalized Anxiety Disorder-7 Scale]). CR was defined as the proportion of items completed in less than 2 seconds per item. We used a multiple linear regression model to examine the effect of individual factors (sex, age, education, smoking, and drinking) on CR across 4 questionnaires. In addition, response times were visualized to identify patterns between careless and careful responders. RESULTS: Females demonstrate lower levels of CR than males when completing the PHQ-9 (β=-.172, 95% CI -0.104 to -0.089; P<.001), PSS (β=-.234, 95% CI -0.162 to -0.14; P<.001), ISI (β=-.207, 95% CI -0.13 to -0.114; P<.001), and GAD-7 (β=-.177, 95% CI -0.108 to -0.093; P<.001). Older participants demonstrated lower levels of CR on the PHQ-9 (β=-.036, 95% CI -0.007 to -0.003; P<.001), ISI (β=-.036, 95% CI -0.007 to -0.003; P<.001), and GAD-7 (β=-.053, 95% CI -0.009 to -0.005; P<.001), but their age was unrelated to CR on the PSS. Interestingly, compared with participants with an associate-level education, those with a high education (bachelor's, master's, or doctoral degree) demonstrated higher levels of CR, especially those with a master's degree (PHQ-9: β=.098, 95% CI 0.136 to 0.188; P<.001 and GAD-7: β=.091, 95% CI 0.125 to 0.178; P<.001). Smokers exhibited varied patterns, with current smokers demonstrating lower levels of CR on the PHQ-9 (β=-.022, 95% CI -0.064 to -0.016; P=.001) and GAD-7 (β=-.014, 95% CI -0.051 to -0.002; P=.03), whereas occasional smokers demonstrated higher levels of CR on the PSS (β=.019, 95% CI 0.010 to 0.050; P=.003) than nonsmokers. Drinkers demonstrated lower levels of CR than nondrinkers, with the strongest effect among occasional drinkers on the PHQ-9 (β=-.163, 95% CI -0.103 to -0.087; P<.001). Analysis of response times revealed that participants tended to spend less time on PHQ-9 and GAD-7 surveys, and CR on PSS and ISI surveys was characterized by skipping questions. CONCLUSIONS: The effect of individual factors on CR varies across questionnaire types. These findings offer valuable insights for questionnaire designers and administrators, highlighting the need for targeted intervention.