Abstract
PURPOSE: Optical stylets are increasingly used by anesthesiologists to effectively manage difficult airways in ear, nose and throat (ENT) surgeries; however, no dedicated assessment tool for this technique currently exists. This study aims to develop and test the validity and reliability of a new objective structured assessment of technical skill (OSATS) tool for optical stylet tracheal intubation. PARTICIPANTS AND METHODS: A new OSATS-based tool for optical stylet intubation was developed at the Airway Management Center of Fudan University, Shanghai, China. A total of 30 anesthesiologists were recruited and divided into three groups based on prior experience with optical stylets: novice (0-5 procedures), intermediate (11-50), and expert (≥ 100), with 10 in each group. Each participant performed basic optical stylet intubation on a patient with normal airway in a clinical setting. The tool was assessed using video recordings to evaluate internal consistency, intrarater and interrater reliability, and construct validity. Construct validity was evaluated using a known-groups comparison approach, based on predefined levels of prior optical stylet experience (novice, intermediate, expert). Two raters independently assessed the videos, with one rater performing a repeat assessment to evaluate intrarater reliability. RESULTS: Cronbach's alpha was 0.92, indicating high internal consistency. The intraclass correlations (ICCs) for intrarater and interrater reliability were 0.97 and 0.94, and the mean kappa coefficients were 0.84 and 0.72, respectively. Mean total scores differed significantly across the three groups: novice (10.50 ± 3.24), intermediate (18.30 ± 4.12), and expert (25.95 ± 2.06) (P < 0.001), demonstrating construct validity through known-groups comparison. CONCLUSION: This study demonstrates that the scoring tool has satisfactory reliability and validity for assessing basic optical stylet intubation skills, with the ability to discriminate between different levels of expertise. This scoring tool may serve as a useful formative assessment tool for anesthesiologists in training.