Real-time detection of kiwifruit flower and bud simultaneously in orchard using YOLOv4 for robotic pollination

•Kiwifruit flower and bud were labeled as 2 classes based on their growth stages.•YOLOv3 and YOLOv4 were applied for flower and bud detection.•YOLOv4 achieved the highest AP of 98.6% on bud detection and 4.6% higher than YOLOv3.•YOLOv4 reached mAP of 97.6% on kiwifruit flower and bud detection in 38.6 ms per image.•YOLOv4 showed good generalizability on another image dataset previously unseen. Robotic pollination may help with reducing high labor requirements and saving expensive pollen on artificial pollination of kiwifruit. Fast and accurate detection of kiwifruit flower and bud simultaneously in an orchard is essential for robotic pollination. It not only makes precision pollination possible but also can predict blooming peak to estimate the optimal pollination timing. However, only kiwifruit flower has been labeled and detected in recent studies. Therefore, kiwifruit flower and bud were labeled, trained, and detected simultaneously for robotic pollination. Well-known YOLOv3 and recently released YOLOv4 were applied to do transfer learning for kiwifruit flower and bud detection. Both were trained in the same image dataset and compared by Average Precision (AP) and processing speed, which were aimed to find a better model. Results showed that mean AP (mAP) of YOLOv4 (97.61%) was higher than YOLOv3 (95.24%) on kiwifruit flower and bud detection. The AP of flower and bud detection achieved by YOLOv4 were 96.66% and 98.57%, respectively, which were 0.17% and 4.58% higher than that of YOLOv3. The detection speed of YOLOv4 was 38.64 ms per image with 4608 × 3456 pixels, which was resized to 608 × 608 pixels in detection. In addition, another image dataset was collected from different years and locations to demonstrate the generalizability of YOLOv3 and YOLOv4, which reported mAPs of 80.98% and 91.49% on them, respectively. It can be concluded that YOLOv4 is promising to achieve real-time detection of kiwifruit flower and bud simultaneously for further flower blooming peak estimation and robotic pollination.