Incorporating the hybrid deformable model for improving the performance of abdominal CT segmentation via multi-scale feature fusion network

•We propose a Hybrid Deformable Model (HDM) consisting of the inter-and intrapatient deformation for more effective data augmentation.•The proposed HDM achieves higher accuracy of the results, indicating that it may potentially become a generalized technique for deep learning-based segmentation, even for image registration and reconstruction.•We fused the multi-scale features into a 3-D attention U-Net to effectively probe the high-level features.•We carry out comparison experiments for multi-organ segmentation and outperformed state-of-the-art methods. [Display omitted] Automated multi-organ abdominal Computed Tomography (CT) image segmentation can assist the treatment planning, diagnosis, and improve many clinical workflows’ efficiency. The 3-D Convolutional Neural Network (CNN) recently attained state-of-the-art accuracy, which typically relies on supervised training with many manual annotated data. Many methods used the data augmentation strategy with a rigid or affine spatial transformation to alleviate the over-fitting problem and improve the network’s robustness. However, the rigid or affine spatial transformation fails to capture the complex voxel-based deformation in the abdomen, filled with many soft organs. We developed a novel Hybrid Deformable Model (HDM), which consists of the inter-and intra-patient deformation for more effective data augmentation to tackle this issue. The inter-patient deformations were extracted from the learning-based deformable registration between different patients, while the intra-patient deformations were formed using the random 3-D Thin-Plate-Spline (TPS) transformation. Incorporating the HDM enabled the network to capture many of the subtle deformations of abdominal organs. To find a better solution and achieve faster convergence for network training, we fused the pre-trained multi-scale features into the a 3-D attention U-Net. We directly compared the segmentation accuracy of the proposed method to the previous techniques on several centers’ datasets via cross-validation. The proposed method achieves the average Dice Similarity Coefficient (DSC) 0.852, which outperformed the other state-of-the-art on multi-organ abdominal CT segmentation results.