Rapid and non-destructive quality grade assessment of Hanyuan Zanthoxylum bungeanum fruit using a smartphone application integrating computer vision systems and convolutional neural networks

The market for extra-grade Hanyuan Zanthoxylum bungeanum fruit (HZB) products is unfortunately rife with adulterated goods, presenting a pressing challenge that necessitates immediate resolution. Therefore, this study developed a smartphone application integrating computer vision systems (CVS) and convolutional neural networks (CNN) for the identification and grade assessment of HZB. A total of 5360 images of individual ZB, including HZB of different grades and other ZB species, were captured by smartphone and used to train, validate, and test the two classification models, VGG16 and Resnet50, where Resnet50 exhibited better performance (accuracy >97%). This model was further employed with the watershed algorithm to segment and classify ZB clusters, specifically identify the extra-grade HZB with an accuracy 97.67%. Additionally, these models were embedded in a smartphone application, and the application's effectiveness was further validated through rigorous defect rate evaluations. Offering a rapid, non-destructive, and user-friendly solution, this smartphone-based approach marks a significant advancement over traditional methods, making it ideal for both professionals and ordinary consumers, especially in resource-limited settings. •CVS based on a phone camera was utilized to capture and segment ZBs' images.•Two CNN models (VGG16 and Resnet50) were trained and compared.•The application integrated classification and identification models (accuracy >97%).•The application's effectiveness was validated by defect rate evaluations.•The innovative approach could rapidly identify HZB and its grade without destruction.