Evaluation of the prototype Anaesthetic Non‐technical Skills for Anaesthetic Practitioners (ANTS‐AP) system: a behavioural rating system to assess the non‐technical skills used by staff assisting the anaesthetist

Summary This study tested the reliability, validity and usability of a prototype behavioural rating system for the non‐technical skills of assistants working with the anaesthetist. Anaesthetic nurses and operating department practitioners (n = 48) used the prototype Anaesthetic Non‐technical Skills for Anaesthetic Practitioners (ANTS‐AP) system to rate the non‐technical skills of anaesthetic assistants in 12 videos of simulated theatre work. Test–retest reliability was assessed with a sub‐sample (n = 12). The skill categories assessed were ‘situation awareness’, ‘teamwork and communication’ and ‘task management’. The internal consistency for the ratings of elements in categories was acceptable (Cronbach's α of 0.78, 0.77 and 0.69, respectively), with more modest inter‐rater reliability (intraclass correlations for categories 0.54, 0.70, 0.86), test–retest reliability (intraclass correlations 0.68, 0.58, 0.38) and accuracy (weighted kappa 0.39). Most participants considered the system complete (n = 42, 87%), the wording clear (n = 48, 100%) and the system useful for structuring observation (n = 48, 100%).