Abstract
BACKGROUND: The development of undergraduate medicine licensure examinations is critical for ensuring physicians’ competencies. Existing literature primarily focuses on exams Psychometric properties while leaving a gap in understanding the developmental processes, stakeholder engagement, and contextual influences, particularly in resource-limited settings. This study evaluated the mechanisms and stakeholders’ perspectives on shaping the Ethiopian undergraduate medicine licensure examination development and implementation practices. METHODS: A qualitative study design combining a phenomenological orientation with a realistic evaluation approach was employed. Purposive sampling was used to select eight experienced experts from four major stakeholder institutions. Data were collected through key informant interviews and cross-validated against exam development guidelines and expert profile datasets. Thematic analysis was conducted to present challenges, enablers, and institutional factors. RESULTS: The study evidenced that the Ethiopian Undergraduate Medicine Licensure Examination follows a standard approach, consisting of task analysis, blueprint preparation, item construction and appraisal, and exam administration. However, critical gaps were identified in expertise psychometric competencies, and challenges associated with paper-based testing and exam security concerns. CONCLUSION: While participants perceive the undergraduate medicine licensure examination design aligned with national goals, its implementation is compromised by technical and logistic gaps. Transitioning to Computer-Based Testing, professionalizing item writing, and balancing stakeholder representation with experienced experts are essential facets to maintain the exam’s validity.