Abstract
Early detection of adverse events and fall injuries may improve patient safety outcomes for clinical trials in geriatric populations. This study evaluates multimodal models combining structured SOAP notes and remote biophysical sensor measurements to classify adverse event occurrences and fall events in a clinical trial with rural older adults participants. XGBoost classifiers were trained on BioBERT, BioClinicalBERT and BERT-Uncased SOAP note embeddings, with and without fused sensor features, and compared across control and intervention cohorts. Non-fused embedding features performed best on Subjective notes for adverse event classification from BioClinicalBERT (AUROC=0.89, Recall=0.88) for controls and BioBERT (AUROC=0.86, Recall=0.73) in the intervention arm. Sensor features provided higher discrimination and recall for adverse events in controls (AUROC=0.68, Recall=0.80) than the intervention arm (AUROC=0.57, Recall=0.10). For fall classification, sensor features outperformed embeddings in the control (AUROC=0.87, Recall=0.32) and intervention (AUROC=0.84, Recall=0.12) cohorts. Assessment and Planning note components had significantly lower AUROC across all embedding feature models. Fusing sensor and embedding features resulted in near-perfect performance from Subjective and Objective notes (AUROC=1.0, Recall=1.0), significantly better than non-fused embeddings. Analysis of NER tokens extracted from SOAP notes showed that model performance differences are associated with cohort-specific documentation practices. SOAP contents in the intervention cohort were more patient-focused, with higher word counts in Subjective sections and narrower AUROC confidence intervals, reflecting increased clinical engagement and improved event capture. These results suggest that combining clinical narratives with continuous sensor measurements can improve the prediction of adverse events and fall injuries, which may increase clinical trial safety and reduce the frequency of in-person assessments.