Generalized zero-shot pest and disease image classification based on causal gating model

•The first to propose textual-visual dataset in agriculture disease image field.•Decouple causal and noisy features using structural causal models.•Use bounded manifold on the unit hypersphere to distinguish seen and unseen domain.•SCM-BDC can avoid the collection and labeling of scarce pest and disease images. Images of scarce agricultural pests and diseases categories are often hard to obtain, and there are frequently no visual examples in web search engines. Therefore, it is challenging to build an image classification model based on supervised learning. To address this issue, Generalized Zero-shot Learning (GZSL) based on Generative Adversarial Networks (GANs) offers an effective solution. However, a major challenge of GZSL is that the model tends to overfit on seen class data, causing unseen classes to be frequently misclassified as seen classes. To meet this challenge, we propose a novel Structural Causal Model-based Binary Domain Classifier (SCM-BDC) for generalized zero-shot pest and disease image classification. Our method introduces a Structural Causal Model (SCM) to extract causal features from visual features to reduce the impact of non-causal features that blur the distinction between seen and unseen classes. Furthermore, we use an Angular Linear Layer (ALL) to project class-level attributes and causal features onto the unit hypersphere and identify a boundary for each seen class. During the testing phase, if the similarity between the sample and all seen attributes is less than the corresponding threshold, it is classified as an unseen class; otherwise, as a seen class. Finally, we use a seen classifier and an unseen classifier to predict the corresponding samples, respectively. Extensive experiments on APTV99, ADTV68, AWA1, AWA2, CUB, SUN, and FLO demonstrate that the proposed method can significantly improve the performance of GZSL. For the APTV99 and ADTV68 datasets, our method achieves a 5.2 % and 1.4 % improvement in GZSL classification accuracy over state-of-the-art methods.