A category attention instance segmentation network for four cardiac chambers segmentation in fetal echocardiography

The network structure of our Category Attention Instance Segmentation Network (CA-ISNet). The input image enters the backbone network to generate FPN feature maps P2–P6. Firstly, for each feature map level, we interpolate it to a size of S × S and use it as the input feature map for the category branch and the mask kernel branch. Secondly, we upsample P2–P5 to 1/4 of the original image size and then add them to obtain the fusion of feature maps. It will be used as the input feature map for the mask feature branch and the category attention branch. Thirdly, the mask feature branch's output is convolved by the output of the mask kernel branch to obtain the predicted mask. Finally, we interpolate the output of the category attention branch to a size of S × S. Then we perform the softmax on the interpolated feature map along the direction of the channels and multiply the feature map without the background channel with the category branch's output feature map to get the corrected category confidence. The category of the mask is specified by the corrected category confidence. [Display omitted] •An instance segmentation framework is proposed for cardiac chamber segmentation in fetal echocardiography.•A novel Category Attention Module is designed to correct the instance misclassification and improve the segmentation accuracy.•Experiments on a fetal echocardiography dataset show that our method can achieve superior segmentation performance against state-of-the-art methods. Fetal echocardiography is an essential and comprehensive examination technique for the detection of fetal heart anomalies. Accurate cardiac chambers segmentation can assist cardiologists to analyze cardiac morphology and facilitate heart disease diagnosis. Previous research mainly focused on the segmentation of single cardiac chambers, such as left ventricle (LV) segmentation or left atrium (LA) segmentation. We propose a generic framework based on instance segmentation to segment the four cardiac chambers accurately and simultaneously. The proposed Category Attention Instance Segmentation Network (CA-ISNet) has three branches: a category branch for predicting the semantic category, a mask branch for segmenting the cardiac chambers, and a category attention branch for learning category information of instances. The category attention branch is used to correct instance misclassification of the category branch. In our collected dataset, which contains echocardiography images with four-chamber views