Abstract
BACKGROUND: The most common symptom of atopic dermatitis (AD) is pruritus, which is often exacerbated at night and leads to nocturnal scratching and sleep disturbance. The quantification of nocturnal scratching provides an objective measure, which could be used as a clinical trial endpoint tracking this AD-related behavior. However, it is not clear how digital health technologies (DHTs) intended to measure scratching perform in the real-world environment of patient homes. OBJECTIVE: In this study, we present the analytical validation of 2 DHTs: the GENEActiv wristband with Philips sleep and scratch algorithms ("Philips") and the Emerald radio frequency touchless sensor ("Emerald") to measure nocturnal scratching in adults with AD. METHODS: Thirty-one participants (15 with moderate AD, 11 with mild AD, and 5 healthy volunteers) were enrolled in the study. Nocturnal scratching was assessed by each DHT in the study participant's home environment over a 4-week observation period. Infrared videos were recorded during sleep twice per week and manually annotated for the intended sleep window (total sleep opportunity [TSO]) and scratching events. Human annotations for sleep and scratch measures were used as a reference for comparison with DHTs ("Reference"). Estimated TSO was compared for each DHT to Reference using Bland-Altman analysis. Within-night agreement of DHT-predicted scratching events versus the Reference was assessed by sensitivity, precision, and F1-score. Intraclass correlation was used to compare night-level scratch summaries (scratch duration per hour of TSO and scratch frequency per hour of TSO) between each DHT and the Reference. RESULTS: Characterization of human-annotated scratching revealed a basal level of scratching in both healthy volunteers (13.1 seconds per hour) and in participants with AD on nights when no itch was reported (10.2 seconds per hour). The TSO window was quantified accurately with both DHTs having a mean bias compared with Reference of <30 minutes. The within-night agreement with reference to scratch detection performance resulted in F1-scores at the disease group level ranging from 0.51 to 0.68 for the Emerald DHT and 0.47 to 0.56 for the Philips DHT. The night-level agreement of nocturnal scratch duration and frequency with human raters fell mostly in the moderate-good range of intraclass correlation coefficients (0.5-0.9) in participants with AD and was not significantly lower than the level of agreement between any 2 human raters. CONCLUSIONS: These results support the analytical validity of both DHTs tested for continuous measurement of nocturnal scratching in individuals with AD in the home environment. Opportunities remain for improving the performance of the DHTs tested, especially in the precision of wrist-worn accelerometer scratch detection, to reach human-level performance. Additional data collection in diverse patient populations will be beneficial for practitioners intending to use or improve these tools for quantifying nocturnal scratching behavior.