Abstract
BACKGROUND: To meet the need for competency assessment in thoracic surgery education, we developed and tested an instrument to assess trainees' ability to perform anatomic lung resection for cancer. METHODS: The Thoracic Competency Assessment Tool-Anatomic Resection for Lung Cancer (TCAT-ARC) was developed through a multistep process involving logical analysis, expert review, and simulation-based and clinical pilot testing. Validity evidence was gathered during a 6-month clinical study of trainees performing anatomic lung resections and assessments of practicing surgeons. Feedback was gathered via post-encounter questionnaires. RESULTS: A 35-item instrument was developed and was tested in the clinical validation study. Seven trainees in 4 North American institutions participated and completed 64 anatomic lung resections. Reliability was high (α = 0.93). Interobserver reliability (k = 0.73) and correlation with an existing global competency scale (k = 0.68) were moderately high. Item analysis revealed the most difficult and discriminatory items, which matched well with a conceptual understanding of lung resection. Both trainees and assessors viewed the instrument as highly educationally effective and user-friendly. Practicing surgeons outperformed trainees. CONCLUSIONS: The TCAT-ARC demonstrated early evidence of validity and reliability in assessing performance of anatomic lung resection. The instrument may be most useful early in training and as a means for providing fine-grained formative feedback about which steps have been mastered and which still require improvement. The TCAT-ARC may be used in training programs to aid in the development of trainees' competency and as a part of an aggregate assessment of trainees' overall mastery of the procedure and readiness for independent practice.