Abstract
INTRODUCTION: Educational milestones are now used to assess the developmental progress of all U.S. graduate medical residents during training. Twice annually, each program's Clinical Competency Committee (CCC) makes these determinations and reports its findings to the Accreditation Council for Graduate Medical Education (ACGME). The ideal way to conduct the CCC is not known. After finding that deliberations reliant upon the new milestones were time intensive, our internal medicine residency program tested an approach designed to produce rapid but accurate assessments. MATERIAL AND METHODS: For this study, we modified our usual CCC process to include pre-meeting faculty ratings of resident milestones progress with in-meeting reconciliation of their ratings. Data were considered largely via standard report and presented in a pre-arranged pattern. Participants were surveyed regarding their perceptions of data management strategies and use of milestones. Reliability of competence assessments was estimated by comparing pre-/post-intervention class rank lists produced by individual committee members with a master class rank list produced by the collective CCC after full deliberation. RESULTS: Use of the study CCC approach reduced committee deliberation time from 25 min to 9 min per resident (p < 0.001). Committee members believed milestones improved their ability to identify and assess expected elements of competency development (p = 0.026). Individual committee member assessments of trainee progress agreed well with collective CCC assessments. CONCLUSIONS: Modification of the clinical competency process to include pre-meeting competence ratings with in-meeting reconciliation of these ratings led to shorter deliberation times, improved evaluator satisfaction and resulted in reliable milestone assessments.