Automatic Diagnosis of Organ Health Status by Deep Learning of Tongue Images

In this paper, we apply artificial intelligence technology to tongue diagnosis in Traditional Chinese Medicine and propose a five-division diagnostic method that divides the tongue image into five parts and diagnoses the health condition of the internal organs. Every part of the tongue, including the tip, middle, two sides and root of the tongue, is supposed to reflect the corresponding organ(s), the heart / lung, stomach / spleen, liver / gall, and kidney individually. The five-division diagnostic method first recognizes and extracts the tongue part from a given image using the image recognition technology of artificial intelligence. Next, the extracted image is divided into the five parts of the tongue. Further, the two sides of the tongue are integrated into one. Finally, the corresponding internal organs' health condition is diagnosed for each part's image. In this paper, Mask R-CNN is used for tongue recognition and tongue image extraction, and five image recognition models (LeNet, ResNet50, ResNet101, DenseNet169, and EfficientNet-B0) and an ensemble of four of these models (except EfficientNet-B0) are used for the health examination of the organs. Experimental results show that ensemble learning is superior to the accuracy of the four individual models and the accuracy rates for all five parts of the tongue reach more than 80%. In addition, comparison experiment using EfficientNet-B0 was conducted in order to compare its accuracy with that of the other four models and as the result EfficientNet-B0 was inferior to any one of the four. From these results, the five-division diagnostic method with the ensemble is effective to diagnoses the health condition of the internal organs.