P113 A detailed analysis of multicentric sleep staging inter-rater variabilities

Abstract Introduction Correct identification of sleep stages is important in sleep disorder diagnosis; but is subject to high inter-scorer variability. Thus, we aimed to systematically analyse sleep staging agreements between ten scorers in a multicentric environment to better understand areas that possess high scoring variations. Method Polysomnographic recordings of 50 individuals (mean (±SD) AHI 12.0±13.2) were independently scored by ten sleep technologists from seven sleep centres ( Europe six, Australia one). The epoch-based majority sleep stage was identified from all scorings; then the mean scorer agreement with this majority score was calculated. We also investigated how sleep-state transitions affect the mean scorer agreement. Results In total, 48556 epochs were analysed. The mean (±SD) scorer agreement with majority score was 86.3±2.9% (Randolph’s kappa κ=0.74) overall; and 86.3±9.5% (κ=0.74), 65.6±13.5% (κ=0.39), 86.1±6.2% (κ=0.74), 88.8±11.2% (κ=0.78), and 91.7±4.5% (κ=0.83) for Wake, N1, N2, N3, and REM respectively. Across the ten scorers, 20.0% of epochs were unanimously scored as N2, and the most frequent disagreement was between N2 and N3 (19.7% of epochs). Only 0.3% of epochs were unanimously scored as N1. There was a negative association between the sleep stage transition frequency of the majority score and the mean scorer agreement (r=-0.61). Discussion High mean scorer agreements were obtained with the majority score except for N1. Surprisingly, considerable disparities were observed among scorers in distinguishing N2 and N3; this may reflect both the discrepancy associated with N2/N3 transitions and potentially different scoring perceptions. Greater scoring discrepancies were identified in individuals with more frequent sleep-state transitions.