Automated Detection of Anatomical Landmarks During Colonoscopy Using a Deep Learning Model

Abstract Background and aims Identification and photo-documentation of the ileocecal valve (ICV) and appendiceal orifice (AO) confirm completeness of colonoscopy examinations. We aimed to develop and test a deep convolutional neural network (DCNN) model that can automatically identify ICV and AO, and differentiate these landmarks from normal mucosa and colorectal polyps. Methods We prospectively collected annotated full-length colonoscopy videos of 318 patients undergoing outpatient colonoscopies. We created three nonoverlapping training, validation, and test data sets with 25,444 unaltered frames extracted from the colonoscopy videos showing four landmarks/image classes (AO, ICV, normal mucosa, and polyps). A DCNN classification model was developed, validated, and tested in separate data sets of images containing the four different landmarks. Results After training and validation, the DCNN model could identify both AO and ICV in 18 out of 21 patients (85.7%). The accuracy of the model for differentiating AO from normal mucosa, and ICV from normal mucosa were 86.4% (95% CI 84.1% to 88.5%), and 86.4% (95% CI 84.1% to 88.6%), respectively. Furthermore, the accuracy of the model for differentiating polyps from normal mucosa was 88.6% (95% CI 86.6% to 90.3%). Conclusion This model offers a novel tool to assist endoscopists with automated identification of AO and ICV during colonoscopy. The model can reliably distinguish these anatomical landmarks from normal mucosa and colorectal polyps. It can be implemented into automated colonoscopy report generation, photo-documentation, and quality auditing solutions to improve colonoscopy reporting quality. Lay Summary Colon cancer is a type of cancer in the large intestine. It is an important health problem in Canada. Colonoscopy is a medical test that uses a camera to see the inside of the large intestine. When looking in the large intestine with this camera, doctors can find and remove abnormal tissue and prevent cancer. So, colonoscopy could help doctors decrease cancer in the large intestine and its related deaths. To find abnormal tissue, you need to see the entire large intestine, from the beginning to the end. We developed a system that uses computer mathematical power to tell us if a doctor could see the entire large intestine. This can be achieved by the spotting of specific items by this system. We found that this system was good at identifying the specific items. Therefore, a doctor can be sure that he saw th