Abstract
OBJECTIVE: To analyse the interrater reliability of the NEUMOBACT checklist and verify whether consistent results are reproducible. METHODS: A validation study with a cross-sectional design, compliant with the GRRAS checklist, among ICU nurses attending a SIMULAZERO course with an Objective Structured Clinical Evaluation simulation format, to verify transfer from theory to clinical practice of knowledge and skills in ventilator-associated pneumonia (VAP) and catheter-related bacteraemia (CRB) prevention. A minimum sample size of 111 pairs of nurse raters was calculated. Interrater agreement was analysed using Gwet's AC1 for each item and as a total for each of the three checklists in the NEUMOBACT instrument. RESULTS: A total of 95 pairs of valid NEUMOBACT checklists were completed by 190 raters with a median age of 29 [25-35] years, 93.7% were female. At the Central Venous Catheter insertion station, Gwet's AC1 was 0.934 (95% CI [0.919-0.949]). Only 2 of the 17 items scored below 0.9. At the Endotracheal Suctioning station, Gwet's AC1 was 0.869 (95% CI [0.851-0.886]). Of the 26 items that made up this station, 16 had an agreement percentage above 0.9, a further 9 were between 0.821 and 0.884, and item 13 had a value of 0.789. At the Patient Care station, Gwet's AC1 was 0.911 (95% CI [0.896-0.927]). Of the 21 items, 17 showed an agreement percentage above 0.9 and 4 were between 0.810 and 0.894. CONCLUSIONS: The interrater reliability of the NEUMOBACT checklist shows substantial agreement between pairs of raters and is therefore validated in this large sample of ICU nurses. RELEVANCE TO CLINICAL PRACTICE: The NEUMOBACT checklist can be useful for assessing skills before and after training in VAP and CRB prevention measures and during debriefing (post-simulation feedback), to reinforce the scientific evidence behind actions and decisions for items that have been performed incorrectly, thus consolidating training already received.