1340 Validation of artificial intelligence algorithm classifying arteriovenous access aneurysms in hemodialysis: a blinded multicenter prospective study

Abstract Background and Aims In hemodialysis (HD) patients, arteriovenous (AV) access aneurysms may lead to severe and potentially life-threatening consequences, such as rupture. To address this issue, we developed an artificial intelligence-based application that utilizes images of the AV access to classify AV aneurysms [1]. We conducted a blinded, multicenter, prospective pilot study to assess the correlation between the classification results generated by the aneurysm classification application and the independent clinical examination conducted by physicians specializing in vascular access care. Method AV accesses were photographed using tablets. These images were then uploaded to the cloud, where they were classified as either "Advanced" or "Not Advanced" by a convolutional neural network algorithm [1] (Fig. 1A). The study compared AV aneurysm classifications generated by our application to those made by vascular access physicians blinded to the app results. The physicians’ classifications served as the ground truth for our analysis. Results The study was conducted at two vascular access care centers in New York, NY, USA. A total of 121 patients were included in the analysis (Table 1). The physicians’ assessment identified a 21% prevalence of advanced aneurysms. The application accurately classified 84 out of 95 aneurysm images as “Not Advanced” and 20 out of 26 as “Advanced,” resulting in an accuracy of 86.0% (95% CI: 78.5% to 91.6%), a sensitivity of 76.9% (95% CI: 56.4% to 91.0%), a specificity of 88.4% (95% CI: 80.2% to 94.1%), and an area under the receiver operating characteristics curve (AUROC) of 0.87 (95% CI: 0.78 to 0.94) (Fig. 1B). Conclusion Our results demonstrate that an AI-powered application exhibits actionable accuracy in classifying AV aneurysms within a demographically diverse HD population. These findings align with Zhang et al., where a team of vascular access experts classified 1,093 out of 1,341 (81.5%) images as “Not Advanced” aneurysms and 248 (18.5%) as “Advanced,” resulting in an AUROC of 0.96 in the validation set (n = 402) [1]. The difference in AUROCs can be attributed to the fact that, in Zhang et al., vascular access experts evaluated only AV access images, whereas in the present study, they performed a physical examination of the patient. Our tool may have the potential to support aneurysm monitoring processes, enabling timely detection, decision-making, and interventions.