Automatic strawberry leaf scorch severity estimation via faster R-CNN and few-shot learning

Using deep learning to estimate strawberry leaf scorch severity often achieves unsatisfactory results when a strawberry leaf image contains complex background information or multi-class diseased leaves and the number of annotated strawberry leaf images is limited. To solve these issues, in this paper, we propose a two-stage method including object detection and few-shot learning to estimate strawberry leaf scorch severity. In the first stage, Faster R-CNN is used to mark the location of strawberry leaf patches, where each single strawberry leaf patch is clipped from original strawberry leaf images to compose a new strawberry leaf patch dataset. In the second stage, the Siamese network trained on the new strawberry leaf patch dataset is used to identify the strawberry leaf patches and then estimate the severity of the original strawberry leaf scorch images according to the multi-instance learning concept. Experimental results from the first stage show that Faster R-CNN achieves better mAP in strawberry leaf patch detection than other object detection networks, at 94.56%. Results from the second stage reveal that the Siamese network achieves an accuracy of 96.67% in the identification of strawberry disease leaf patches, which is higher than the Prototype network. Comprehensive experimental results indicate that compared with other state-of-the-art models, our proposed two-stage method comprising the Faster R-CNN (VGG16) and Siamese networks achieves the highest estimation accuracy of 96.67%. Moreover, our trained two-stage model achieves an estimation accuracy of 88.83% on a new dataset containing 60 strawberry leaf images taken in the field, which indicates its excellent generalization ability. •A method was proposed for fine-grained division of strawberry leaf scorch severity.•For complex background problem, Faster R-CNN was used to locate strawberry leaves.•For few-shot problem, Siamese Network was used to identify strawberry patches.•150 images in laboratory and field were used to improve the generalization ability.