Variability in the assessment of advanced life support skills

The number of short ‘life support’ and emergency care courses available are increasing. Variability in examiner assessments has been reported previously in more traditional types of examinations but there is little data on the reliability of the assessments used on these newer courses. This study evaluated the reliability and consistency of instructor marking for the Resuscitation Council UK Advanced Life Support Course. Twenty five instructors from 15 centres throughout the UK were shown four staged video recorded defibrillation tests (one repeated) and three cardiac arrest simulation tests in order to assess inter-observer and intra-observer variability. These tests form part of the final assessment of competence on an Advanced Life Support course. Significant levels of variability were demonstrated between instructors with poor levels of agreement of 52–80% for defibrillation tests and 52–100% for cardiac arrest simulation tests. There was evidence of differences in the observation/recognition of errors and rating tendencies of instructors. Four instructors made a different pass/fail decision when shown defibrillation test 2 for a second time leading to only moderate levels of intra-observer agreement ( κ=0.43). In conclusion there is significant variability between instructors in the assessment of advanced life support skills, which may undermine the present assessment mechanisms for the advanced life support course. Validation of the assessment tools for the rapidly growing number of life support courses is required with urgent steps to improve reliability where required. O número de cursos de suporte de vida e de medicina de emergência, de curta duração, tem vindo a aumentar. Há observações prévias que relatam a variação inter examinadores na avaliação dos cursos tradicionais, mas os dados relativos a estes novos cursos são escassos. Este estudo avalia a fiabilidade e consistência dos procedimentos dos Instrutores nos Cursos de Suporte Avançado de Vida (SAV) do Resuscitation Council UK. Foram exibidos quatro vı́deos de provas de avaliação em desfibrilhação (um deles repetido) três dos quais de situações de simulação de paragem cardı́aca, a vinte e cinco Instrutores de quinze centros dispersos pelo Reino Unido, com a intenção de avaliar a variabilidade das interpretações inter observadores e intra observador. Estes testes são parte da avaliação final dos cursos de SAV. Verificou-se existir um elevado grau de variabilidade inter Instrutores, com baixos