Robust Plant Identification Based on the Combination of Multiple Images and Taxonomic Information

Plant identification poses a challenge for end-users, including novice botanists, due to the high similarity between species, particularly those within the same genus and family, as well as the considerable variation among trees of the same species. To tackle to this challenge, botanists frequently integrate multiple pieces of information about plants to determine their species. Inspired by this observation, in this paper we propose a robust plant identification method that integrate the taxonomic information and multiple images observed from the plant. To achieve this, we define different taxonomic losses to capture the relationship of species within the same genus and of genera within the same family. These loss functions are combined with various deep learning network backbones. Additionally, to leverage the multiple observed images of the plant, we utilize late fusion to merge the identification results obtained for each image. Numerous experiments have been conducted on VietForest - a dataset collected for 156 species in PhuTho province in Vietnam to evaluate the effectiveness of the proposed method. The experimental results demonstrate that the proposed method outperforms the existing methods on VietForest dataset, yielding the accuracy of 84.88%, 82.26% and 81.14% for Family, Genus and Species respectively when using one input image. In multiple image-based plant identification, the proposed method achieves the best results of 97.45%, 91.77% and 89.35% for Family, Genus and Species with five images. Thanks to the superior results achieved by the proposed method, the plant identification module has been integrated into an application dedicated to plant management in PhuTho province.