Deep learning-based lymphocyte infiltration detection on pathological images

Background and purpose: Deep learning methods can be used for automatic segmentation and detection of lymphocytes on pathological images. This study aimed to assess the performance of using variational autoencoding pre-training method for lymphocyte infiltration detection on pathological images, as well as the impact of removing tumor necrosis regions on model performance. Methods: Using variational autoencoding (VAE) pre-training method, pre-training was performed on a large number of unlabeled pathological images from the Cancer Genome Atlas (TCGA) database (TCGA-COAD and TCGA-READ) to obtain an auto-encoding pre-training model, and then a classifier model of lymphocyte infiltration was trained on the public data samples. To avoid confusion with necrotic regions, a Unet segmentation model for tumor necrotic regions was trained to remove the influence of tumor necrotic regions on lymphocyte identification. Results: The lymphocyte infiltration detection model pre-trained with the VAE model had an area under curve (AUC) of 0.979 (95% CI: 0.978-0.980), an accuracy of 92.5% (95% CI: 92.3%-92.6%), a kappa value of 0.849, sensitivity of 0.908, specificity of 0.941, precision of 0.939, recall of 0.908, and F1 of 0.923 under the receiver operating characteristic (ROC) curve on the training set. The AUC for the validation set was 0.968 (95% CI: 0.964-0.972), the accuracy was 91.3% (95% CI: 90.6%-92.0%), kappa value was 0.826, sensitivity was 0.898, specificity was 0.928, precision was 0.925, recall was 0.898, and F1 was 0.912. The results of Resnet18 model on the labeled dataset were as follows: accuracy of the validation set was 83.2 % (95% CI: 82.2%-84.1%), kappa value was 0.664, sensitivity was 0.823, specificity was 0.840, precision was 0.838, recall was 0.823 and F1 was 0.830. The Unet model that segmented the necrotic regions of the tumors had a final DICE of 0.78 for the training set, and 0.76 for the validation. After removing the necrotic region, the predictive performance of the pre-trained lymphocyte infiltration detection model using the VAE proposed in this article was improved to some extent, with the AUC on the validation set increasing from 0.968 (95% CI: 0.964-0.972) to 0.971 (95% CI: 0.968-0.975). The accuracy was 92.4% (95% CI: 91.7%-93.0%), kappa value was 0.849, sensitivity was 0.928, specificity was 0.921, precision was 0.921, recall was 0.928, and F1 was 0.925. Conclusion: Using the variational autoencoding model pre-training method to class