Mask R-CNN-Based Detection and Segmentation for Pulmonary Nodule 3D Visualization Diagnosis

3D visualization diagnosis for pulmonary nodule detection and segmentation is becoming a promising topic in the field of surgical researches and applications. Aiming at assisting radiologists to diagnose pulmonary nodules more accurately, the methods of detection and segmentation for pulmonary nodule 3D visualization diagnosis were proposed based on Mask Region-Convolutional Neural Network (Mask R-CNN) and ray-casting volume rendering algorithm. The Mask R-CNN used resnet50 as the backbone and applied Feature Pyramid Network (FPN) to fully explore multiscale feature maps. And then, Region Proposal Network (RPN) was used to propose candidate bounding boxes. Furthermore, the mask matrices and the raw medical image sequences were multiplied to obtain sequences of predicted pulmonary nodules. Finally, ray-casting volume rendering algorithm was applied to generate the 3D models of pulmonary nodules. The proposed methods are tested and evaluated on publicly available LUNA16 dataset and the independent dataset from Ali TianChi challenge. Experimental results show that Mask R-CNN of weighted loss reaches sensitivities of 88.1% and 88.7% at 1 and 4 false positives per scan, respectively. Meanwhile, we can obtain AP@50 score of 0.882 using Mask R-CNN with weighted loss on labelme_LUNA16 dataset, which outperforms many existing state-of-the-art approaches of detection and segmentation of pulmonary nodules.