Abstract
BACKGROUND: Information exchanged during handoffs contributes importantly to a team's shared mental model. There is no established instrument to measure shared clinical understanding as a marker of handoff quality. OBJECTIVE: To study the reliability, validity, and feasibility of the pediatric cardiology Patient Knowledge Assessment Tool (PKAT), a novel instrument designed to measure shared clinical understanding for pediatric cardiac intensive care unit patients. DESIGN: To estimate reliability, 10 providers watched 9 videotaped simulated handoffs and then completed a PKAT for each scenario. To estimate construct validity, we studied 90 handoffs in situ by having 4 providers caring for an individual patient each complete a PKAT following handoff. Construct validity was assessed by testing the effects of provider preparation and patient complexity on agreement levels. SETTING: A 24-bed pediatric cardiac intensive care unit in a freestanding children's hospital. RESULTS: Video simulation results demonstrated score reliability. Average inter-rater agreement by item ranged from 0.71 to 1.00. During in situ testing, agreement by item ranged from 0.41 to 0.87 (median 0.77). Construct validity for some items was supported by lower agreement rates for patients with increased length of stay and increased complexity. DISCUSSION: Results suggest that the PKAT has high inter-rater reliability and can detect differences in understanding between handoff senders and receivers for routine and complex patients. Additionally, the PKAT is feasible for use in a real-time clinical environment. The PKAT or similar instruments could be used to study effects of handoff improvement efforts in inpatient settings.