Improved Food Region Extraction Using State-of-the-Art Saliency Detection

In this study, the accuracy of food region extraction, which is used to obtain image pixels in the food regions from food images and remove pixels in the background regions such as table and plate, was improved. In the proposed method, state-of-the-art saliency detection, which is used to predict pixels that attract human gaze based on the deep neural network (DNN), and semiautomatic segmentation, which is used to iteratively refine food and background regions by using graph theory, are employed. The comparative experiment demonstrated that the proposed method significantly increased the mean F-measure, which is a comprehensive evaluation metric, over that of conventional saliency-based food region extraction by 9.15% by reducing the erroneous determination of the background as the food region. Furthermore, the F-measure was higher than that of UNet+, which is a DNN-based semantic segmentation trained on a well-known public image dataset. This paper comprehensively details the analysis of performance improvement.