Brain gray matter nuclei segmentation on quantitative susceptibility mapping using dual-branch convolutional neural network

Abnormal iron accumulation in the brain subcortical nuclei has been reported to be correlated to various neurodegenerative diseases, which can be measured through the magnetic susceptibility from the quantitative susceptibility mapping (QSM). To quantitatively measure the magnetic susceptibility, the nuclei should be accurately segmented, which is a tedious task for clinicians. In this paper, we proposed a dual-branch residual-structured U-Net (DB-ResUNet) based on 3D convolutional neural network (CNN) to automatically segment such brain gray matter nuclei. Due to memory limit, 3D-CNN-based methods typically adopted image patches, instead of the whole volumetric image, which, however, ignored the spatial contextual information of the neighboring patches, and therefore led to the accuracy loss. To better tradeoff segmentation accuracy and the memory efficiency, the proposed DB-ResUNet incorporated patches with different resolutions. By jointly using QSM and 3D T1 weighted imaging (T1WI) as inputs, the proposed method was able to achieve better segmentation accuracy over its single-branch counterpart, as well as the conventional atlas-based method and the classical 3D CNN structures. The susceptibility values and the volumes were also measured, which indicated that the measurements from the proposed DB-ResUNet was able to present high correlation with values from the manually annotated regions of interest. •A 3D convolution neural network (CNN) method is proposed for gray matter nuclei segmentation on QSM and 3D T1WI images.•A dual branch CNN structure is proposed to enlarge the fields of view of the network.•Proposed method showed more prominent performance in nuclei segmentation than atlas-based and other deep-learning methods.•The effectiveness on susceptibility value measurement is also discussed, where the proposed method presented better accuracy.