Abstract
BACKGROUND: Structured oral examinations (SOEs) are valid instruments to assess applied knowledge and reasoning skills (competency level 2). They bridge the gap between written examinations focusing on factual knowledge (level 1) and practical formats such as objective structured clinical evaluations (OSCEs) assessing performance skills (from level 3). Through standardized case vignettes, checklists, and defined scoring criteria, SOEs offer potential for enhanced objectivity, reliability, and comparability. OBJECTIVE: Using pure tone audiometry (PTA) and rhinomanometry (RMM) as examples, this study examined the feasibility, psychometric quality (internal consistency, item discrimination, factor analysis), and examiner effects of an SOE designed according to the Nationaler Kompetenzbasierter Lernzielkatalog Medizin (NKLM), competency level 2. METHODS: Within the curricular ENT teaching program, PTA (n = 217) and RMM (n = 190) were taught and subsequently assessed in an SOE. Five categories were rated (identification, performance, description, interpretation, and differential diagnosis; maximum 27 points), with point deductions for examiner assistance. Eight medical examiners conducted the exams in a 1:1 setting. Descriptive statistics and a one-way analysis of variance (ANOVA) were used to analyze examiner differences. RESULTS: The mean total score was 25.20 ± 1.94 for PTA and 24.84 ± 1.94 for RMM. The highest scores were obtained for identification and performance, the lowest for description (PTA) and interpretation (RMM). Significant examiner differences were found for PTA (p = 0.001) but not for RMM (p = 0.078). CONCLUSION: The SOE proved to be a feasible and reliable format for assessing applied knowledge and reasoning. The observed examiner differences highlight the need for examiner training, calibration, and standardized scoring instruments. Structured oral examinations can substantially contribute to implementing competency-based assessment in medical education.