Angiographic results of surgical or endovascular treatment of intracranial aneurysms: a systematic review and inter-observer reliability study

Purpose Results of surgical or endovascular treatment of intracranial aneurysms are often assessed using angiography. A reliable method to report results irrespective of treatment modality is needed to enable comparisons. Our goals were to systematically review existing classification systems, and to propose a 3-point classification applicable to both treatments and assess its reliability. Methods We conducted two systematic reviews on classification systems of angiographic results after clipping or coiling to select a simple 3-category scale that could apply to both treatments. We then circulated an electronic portfolio of angiograms of clipped (n=30) or coiled (n=30) aneurysms, and asked raters to evaluate the degree of occlusion using this scale. Raters were also asked to choose an appropriate follow-up management for each patient based on the degree of occlusion. Agreement was assessed using Krippendorff’s α statistics (α K ), and relationship between occlusion grade and clinical management was analyzed using Fisher’s exact and Cramer’s V tests. Results The systematic reviews found 70 different grading scales with heterogeneous reliability (kappa values from 0.12 to 1.00). The 60-patient portfolio was independently evaluated by 19 raters of diverse backgrounds (neurosurgery, radiology, and neurology) and experience. There was substantial agreement (α K =0.76, 95%CI, 0.67–0.83) between raters, regardless of background, experience, or treatment used. Intra-rater agreement ranged from moderate to almost perfect. A strong relationship was found between angiographic grades and management decisions (Cramer’s V: 0.80±0.12). Conclusion A simple 3-point scale demonstrated sufficient reliability to be used in reporting aneurysm treatments or in evaluating treatment results in comparative randomized trials.