Automated segmentation for whole human eye OCT image using RM multistage Mask R-CNN

Accurate segmentation of structures in whole human eye optical coherence tomography (OCT) images can improve the accuracy of disease diagnosis and ophthalmic biological parameter measurement. However, due to the complex features of the cornea, lens, and retina in the whole human eye OCT image and external interference, it is difficult to precisely extract the object features, which restricts the segmentation accuracy of whole human eye OCT images. A relation module (RM) multistage Mask Region-based Convolutional Neural Network (R-CNN) method for whole human eye OCT image segmentation is established. Based on Mask R-CNN, modulated deformable convolution is employed to produce an irregular receptive field and improve the adaptability of the network to the whole human eye OCT image’s object deformation. In the feature map extraction, the RM is combined to learn the position relation feature map of the human eye structures, which is utilized to enhance feature maps. A multistage mask is constructed in the segmentation branch, and the error is corrected through iterations of the mask to improve the segmentation accuracy of the whole human eye OCT image. Combined with the above structures, the RM multistage Mask R-CNN method is constructed for whole human eye OCT image segmentation. The model is trained by whole human eye OCT images and is applied to realize highly accurate whole human eye OCT image segmentation. Comparison experiments with K-means, U-net, and Feature Pyramid Networks (FPN)-deformable-mask R-CNN are performed to verify the segmentation accuracy and robustness of the proposed method to complex deformation and external interference.