Abstract
BACKGROUND: Multi-source evaluation has demonstrated value for trainees, but is not generally provided to residency or fellowship program directors (PDs). OBJECTIVE: To develop, implement, and evaluate a PD multi-source evaluation process. METHODS: Tools were developed for PD evaluation by trainees, department chairs, and graduate medical education (GME) leadership. Evaluation questions were based on PD responsibilities, including Accreditation Council for Graduate Medical Education (ACGME) requirements. A follow-up survey assessed the process. RESULTS: Evaluation completion rates were as follows: trainees in academic year 2012-2013, 53% (958 of 1824), and in academic year 2013-2014, 42% (800 of 1898); GME directors in 2013-2014, 100% (95 of 95); and chairs/chiefs in 2013-2014, 92% (109 of 118). Results of a follow-up survey of PDs (66%, 59 of 90) and chairs (74%, 48 of 65) supports the evaluations' value, with 45% of responding PDs (25 of 56) and 50% of responding chairs (21 of 42) characterizing them as "extremely" or "quite" useful. Most indicated this was the first written evaluation they had received (PDs 78%, 46 of 59) or provided (chairs 69%, 33 of 48) regarding the PD role. More than 60% of PD (30 of 49) and chair respondents (24 of 40) indicated trainee feedback was "extremely" or "quite" useful, and nearly 50% of PDs (29 of 59) and 21% of chairs (10 of 48) planned changes based on the results. Trainee response rates improved in 2014-2015 (52%, 971 of 1872) and 2015-2016 (69%, 1276 of 1837). CONCLUSIONS: In our institution, multi-source evaluation of PDs was sustained over 4 years with acceptable and improving evaluation completion rates. The process and assessment tools are potentially transferrable to other institutions.