A Computer Vision Solution to Cross-cultural Food Image Classification and Nutrition Logging

The US is a culturally and ethnically diverse nation, which brings with it a wide array of cuisines and eating habits that extend far beyond traditional Western foods. Each of these cuisines has its own nutritional benefits and drawbacks, influencing human health in various ways. There is a growing need for people to access the nutritional profiles of their diverse daily meals to better manage their health. A promising approach to democratize food image classification and nutritional logging is the use of deep learning to extract this information from images provided by users. However, current computer vision applications—a subspecialty of deep learning used for food classification—are often limited by their reliance on Western-biased datasets. Additionally, creating a diverse image dataset for training computational models is challenging due to the vast number of global cuisines. Clearly, there is a need for a pipeline capable of continuously learning to predict new categories of foods. In this project, we propose to design an adaptable prototype pipeline using hierarchical neural networks that can classify international food images, even those the model has rarely or never seen before.