A real-time vision guidance method for autonomous longan picking by the UAV

•A real-time vision guidance method for UAV-based longan picking is proposed.•C2F-Dense enhances feature extraction in longan cluster detection.•A novel MS-OKS loss improves the accuracy of picking points detection.•The model achieves lightweight with high detection accuracy by channel pruning.•Experimental results show the superiority of the proposed method. The unmanned aerial vehicle (UAV) with high flexibility has great application prospect in autonomous longan picking. However, the robustness of the UAV preplanned flight mode is difficult to satisfy precise picking requirements. This study proposes a novel method for longan autonomous picking based on the UAV with real-time visual guidance. The correlations between the UAV velocity and positions of the longan cluster and picking point are established, and make the UAV dynamically adjusts its flight velocity to approach the cluster and accurately align the point for autonomous picking. The longan cluster and their picking points positions are obtained via the YOLOv8n improved by C2F-Dense, MS-OKS Loss and channel pruning. The improved model achieves high accuracy detection, low computation cost and real-time response capabilities. The experimental results revealed that the APmax50 of the improved YOLOv8n for longan cluster detection reached 74.30 %, and the FPS increased by 40.57 %. For picking point detection, the APmax85:95 reached 86.68 % with 88.77 % FPS increase. Additionally, the proposed method has been tested on the DJI M300 without GPU, and its reliability was verified. This research provides technical support for autonomous picking.