Automatic Detection of High-Uptake Lesions in Oncologic FDG PET using faster R-CNN Deep Learning Model

Introduction: Review of oncologic FDG PET images normally includes the processes of optimizing the view window, location in coronal MIP, synchronized review other modality images such as (CT / MRI) in transversal plane, size measurement and lesion counting, which takes much labor effort. Deep learning is an emerging technique which allows the automatic detection and segmentation of lesions or organs, and might accelerate the review of oncologic FDG PET images and enable the improved characterization of lesions. In study, we are aiming to develop an automatic high-uptake lesion detection as a first step approach to fast review of oncologic FDG PET images. Methods: The deep learning model we explored in this study was a 2D faster R-CNN trained in Coco image datasets and fine-tuned by 840 patients' annotated coronal FDG PET MIP images. The input matrix size was modified to 128 x 128 with 1 channel and output 7 classes (5 classes of normal FDG high-uptake organs including brain, thyroid, heart, renal and bladder, other 2 classes representing various markers and high-uptake lesions) and corresponding bounding boxes as detailed in Figure 1. Among the 840 MIP images, 820 were collected from publications and internet via key words search, where the high-uptake lesions were localized by referring to the image descriptions and contoured using a 2D bounding box; the other 20 were acquired in our imaging center. For the validation of this lesion detection model, another 60 patients' FDG PET MIP images in our imaging center were annotated and enrolled into this study as testing sets. Either the training or the testing images were annotated by a 5-year experienced nuclear medicine physician. The pre-processing steps include image normalization, sliding-window in z direction, resizing. The post-processing step was ROI union. During the training phase, the images were augmented on-the-fly using mirror, scale, random inserting various markers. In total, 6 times training images were used to train the model. Confusion matrix analysis were explored to evaluate the performance of the trained model in a multi-class classification and binary classification wise. The accuracy, sensitivity, specificity, positive predicative value (PPV) and negative predictive value (NPV) were reported for binary classification. Results: The disease spectrum of training datasets enrolled into this study consisted of head-neck cancer, lung cancer, esophageal cancer, lymphoma, whole body metastasis,