Abstract
BACKGROUND AND OBJECTIVES: Entrustable professional activities (EPAs) have recently been defined for neurocritical care (NCC) advanced practice providers (APPs). There is no available assessment measure for APPs in neurology or NCC with supporting validity evidence. We aimed to assess the relationship between supervisor-assessed entrustment, performance in simulated medical and neurologic emergencies, and self-assessed entrustment among NCC APPs. METHODS: This was a simulation quality improvement study. Participants were NCC APPs at 5 academic medical centers in the United States between April and November 2024. Participants completed 2 simulated scenarios designed to assess performance in the management of medical and neurologic emergencies. The primary outcome was the significance of the relationship between checklist-based performance in simulated scenarios and supervisor-assessed entrustment among NCC APPs. Secondary outcomes were the agreement between supervisor-assessed and self-assessed entrustment among NCC APPs. RESULTS: There was a significant positive association between physician supervisor-assessed entrustment and performance on the critical action checklist (r(s) = 0.57, p = 0.002). There was no significant correlation between performance in the simulated scenarios and APP supervisor-assessed entrustment (r(s) = -0.02, p = 0.952). There was no difference between overall EPA physician supervisor assessment, APP supervisor assessment, and self-assessment (5-point entrustment-supervision scale median [interquartile range (IQR)], 4 [3-5] vs 4 [4-5] vs 4 [4-4], p = 0.598). There was substantial agreement in 490 discreet physician and self-assessed EPAs (κ(w) = 0.62). Among 322 individual EPAs that were self-assessed and assessed by an APP supervisor, there was fair agreement (κ(w) = 0.38). There was a significant positive association between physician supervisor-assessed entrustment and participant NCC experience (r(s) = 0.85, p < 0.001). DISCUSSION: Physician supervisor EPA-based assessment was positively correlated with NCC APP performance in 2 simulated neurologic and medical emergencies, providing validity evidence for the use of EPA-based assessments as a component of competency-based medical education. There was substantial agreement between physician supervisor and APP self-assessment of EPA-based entrustment, highlighting the potential to incorporate self-assessment in the planning of NCC APP onboarding and ongoing educational curricula.