Slice imputation: Multiple intermediate slices interpolation for anisotropic 3D medical image segmentation

We introduce a novel frame-interpolation-based method for slice imputation to improve segmentation accuracy for anisotropic 3D medical images, in which the number of slices and their corresponding segmentation labels can be increased between two consecutive slices in anisotropic 3D medical volumes. Unlike previous inter-slice imputation methods, which only focus on the smoothness in the axial direction, this study aims to improve the smoothness of the interpolated 3D medical volumes in all three directions: axial, sagittal, and coronal. The proposed multitask inter-slice imputation method, in particular, incorporates a smoothness loss function to evaluate the smoothness of the interpolated 3D medical volumes in the through-plane direction (sagittal and coronal). It not only improves the resolution of the interpolated 3D medical volumes in the through-plane direction but also transforms them into isotropic representations, which leads to better segmentation performances. Experiments on whole tumor segmentation in the brain, liver tumor segmentation, and prostate segmentation indicate that our method outperforms the competing slice imputation methods on both computed tomography (1\% Dice improvement for CT liver tumor segmentation) and magnetic resonance images volumes (over 2\% Dice improvement for MRI prostate segmentation) in most cases. •We use the idea of frame interpolation to solve the anisotropy problem of 3D medical volumes for medical image segmentation.•We use a smoothness loss function to improve slice smoothness of 3D medical volumes in the through-plane direction.•We employ a multitask learning mechanism to improve the authenticity of the interpolated slices.