Convolutional neural network‐based pelvic floor structure segmentation using magnetic resonance imaging in pelvic organ prolapse

Purpose Automated segmentation could improve the efficiency of modeling‐based pelvic organ prolapse (POP) evaluations. However, segmentation performance is limited by the blurry soft tissue boundaries. In this study, we aimed to present a hybrid solution for uterus, rectum, bladder, and levator ani muscle segmentation by combining a convolutional neural network (CNN) and a level set method. Methods We used 24 sagittal pelvic floor magnetic resonance (MR) series from six anterior vaginal prolapse and six posterior vaginal prolapse subjects (a total 528 MR images). The stress MR images were performed both at rest and at maximal Valsalva. We assigned 264 images for training, 132 images for validation, and 132 images for testing. A CNN was designed by introducing a multi‐resolution features pyramid module (MRFP) into an encoder‐decoder model. Depth separable convolution and pretraining were used to improve model convergence. Multiclass cross entropy loss and multiclass Dice loss were used for model training. The dice similarity coefficient (DSC) and average surface distance (ASD) were used for evaluating the segmentation results. To prove the effectiveness of our model, we compared it with advanced segmentation methods including Deeplabv3+, U‐Net, and FCN‐8s. The ablation study was designed to quantify the contributions of MRFP, the encoder network, and pretraining. Besides, we investigated the working mechanism of MRFP in the segmentation network by comparing our model with three of its variants. Finally, the level set method was used to improve the CNN model further. Results Dice loss showed better segmentation performance than multiclass cross entropy loss. MRFP was efficacious for different encoder networks. With MRFP, U‐Net and U‐Net‐X (X represents Xception encoder network) have improved the DSC, on average by 6.8 and 5.3 points. Compared with different CNN models, our model achieved the highest average DSC of 65.6 points and the lowest average ASD of 2.9 mm. With the level set method, the DSC of our model improved to 69.4 points. Conclusions MRFP proved to be effective in addressing the blurry soft tissue boundary problem on pelvic floor MR images. A hybrid solution based on CNN and level set method was presented for pelvic organ segmentation both at rest and at maximal Valsalva; with this method, we achieved state‐of‐the‐art results.