Early CT changes in patients admitted for thrombectomy: Intrarater and interrater agreement

OBJECTIVE:To systematically review the literature and assess agreement on the Alberta Stroke Program Early CT Score (ASPECTS) among clinicians involved in the management of thrombectomy candidates. METHODS:Studies assessing agreement using ASPECTS published from 2000 to 2015 were reviewed. Fifteen raters reviewed and scored the anonymized CT scans of 30 patients recruited in a local thrombectomy trial during 2 independent sessions, in order to study intrarater and interrater agreement. Agreement was measured using intraclass correlation coefficients (ICCs) and Fleiss kappa statistics for ASPECTS and dichotomized ASPECTS at various cutoff values. RESULTS:The review yielded 30 articles reporting 40 measures of agreement. Populations, methods, analyses, and results were heterogeneous (slight to excellent agreement), precluding a meta-analysis. When analyzed as a categorical variable, intrarater agreement was slight to moderate (κ = 0.042–0.469); it reached a substantial level (κ > 0.6) in 11/15 raters when the score was dichotomized (0–5 vs 6–10). The interrater ICCs varied between 0.672 and 0.811, but agreement was slight to moderate (κ = 0.129–0.315). Even in the best of cases, when ASPECTS was dichotomized as 0–5 vs 6–10, interrater agreement did not reach a substantial level (κ = 0.561), which translates into at least 5 of 15 raters not giving the same dichotomized verdict in 15% of patients. CONCLUSIONS:In patients considered for thrombectomy, there may be insufficient agreement between clinicians for ASPECTS to be reliably used as a criterion for treatment decisions.