Deep vessel segmentation by learning graphical connectivity

•A graph neural network (GNN) can learn global vascular structures in medical images.•A CNN only learns local appearances on the regular image grid and thus can be limited.•The vessel graph network (VGN) combines a GNN into a comprehensive CNN architecture.•The VGN jointly learns both local appearance and global structure of vessels.•The VGN structure can be applied to any existing CNN structure to improve accuracy. [Display omitted] We propose a novel deep learning based system for vessel segmentation. Existing methods using CNNs have mostly relied on local appearances learned on the regular image grid, without consideration of the graphical structure of vessel shape. Effective use of the strong relationship that exists between vessel neighborhoods can help improve the vessel segmentation accuracy. To this end, we incorporate a graph neural network into a unified CNN architecture to jointly exploit both local appearances and global vessel structures. We extensively perform comparative evaluations on four retinal image datasets and a coronary artery X-ray angiography dataset, showing that the proposed method outperforms or is on par with current state-of-the-art methods in terms of the average precision and the area under the receiver operating characteristic curve. Statistical significance on the performance difference between the proposed method and each comparable method is suggested by conducting a paired t-test. In addition, ablation studies support the particular choices of algorithmic detail and hyperparameter values of the proposed method. The proposed architecture is widely applicable since it can be applied to expand any type of CNN-based vessel segmentation method to enhance the performance.