Study on strategy of CT image sequence segmentation for liver and tumor based on U-Net and Bi-ConvLSTM

•A branch structure is added on the skip connections of U-net to supplement features.•Bi-ConvLSTM based post-processing for U-net leads to more accurate contours.•Integrating sequence information into U-net achieves better coincidence degree.•Sequence segmentation hardly increases model parameter number compared with U-net. Accurate segmentation of the liver and tumors in computed tomography (CT) images is critical for intelligent computer-aided diagnosis (CAD). The commonly used segmentation methods based on fully convolutional networks (FCN) only take a single image into consideration but do not make good use of sequence information. In this paper, two more feasible sequence segmentation strategies than 3D U-net which can utilize inter-slice and intra-slice features simultaneously at the lower hardware and time cost are studied to improve the segmentation result. U-net serves as the backbone model of segmentation and Bi-directional convolutional long short-term memory (Bi-ConvLSTM) is chosen to extract and fuse the inter-slice feature. Strategy A corrects the pre-segmented results of U-net in the fusion of sequence information as a post-processing, where Mod-1, Mod-2 and Mod-3 models are built to compare the effects of width, depth, and residual structure on the modified model of sequence segmentation. Strategy B directly integrates the fusion of sequence information into the feature extraction of U-net, and then an end-to-end model called W-net is built based on it. The experiment results show that both strategies improve the liver and tumor segmentation performance in various aspects. The results based on strategy A are closer to the ground truth with less misdiagnose region: Mod-1 achieves better accuracy on liver contour segmentation because of the largest model width; Mod-2 can obtain more accurate tumor contour since the greatest depth of feature extraction process; and Mod-3 is at the average segmentation performance. Therefore, strategy A is recommended in the application of surgery planning of tumors. Strategy B achieves better space coincidence degree and less training time cost, which is more suitable for the early screening of liver cancer.