Efficient extraction of deep image features using convolutional neural network (CNN) for applications in detecting and analysing complex food matrices

The development of techniques and methods for rapidly and reliably detecting and analysing food quality and safety products is of significance for the food industry. Traditional machine learning algorithms based on handcrafted features normally have poor performance due to their limited representation capacity for complex food characteristics. Recently, the convolutional neural network (CNN) emerges as an effective and potential tool for feature extraction, which is considered the most popular architecture of deep learning and has been increasingly applied for the detection and analysis of complex food matrices. In the current review, the structure of CNN, the method of feature extraction based on 1-D, 2-D and 3-D CNN models, and multi-feature aggregation methods are introduced. Applications of CNN as a depth feature extractor for detecting and analyzing complex food matrices are discussed, including meat and aquatic products, cereals and cereal products, fruits and vegetables, and others. In addition, data sources, model architecture and overall performance of CNN with other existing methods are compared, and trends of future studies on applying CNN for food detection and analysis are also highlighted. CNN combined with nondestructive detection techniques and computer vision system show great potential for effectively and efficiently detecting and analysing complex food matrices, and the features based on CNN show better performance and outperform the features handcrafted or those extracted by machine learning algorithms. Although there still remains some challenges in using CNN, it is expected that CNN models will be deployed on mobile devices for real-time detection and analysis of food matrices in future. •The principle and architectures of convolutional neural network are presented.•Feature extraction methods based on convolutional neural network are investigated.•Recent applications of convolutional neural network in food are summarized.•Challenges and future work of convolutional neural network are discussed.