Efficient detection and picking sequence planning of tea buds in a high-density canopy

•A cutting-edge network YOLOX-S was deployed to detect tea buds in high-density canopy.•An improved pointer network (PtrNet) was proposed to find the optimal solution to the picking sequence planning problem.•YOLOX-S only needed 17.43 ms to process a single image with an average precision of 0.642.•The improved pointer network only required 1.69 ms to output an optimal path for the problem up to 100 tea buds. Real-time robotic tea picking ensures the economic benefits of the well-known high-quality tea industry. Efficient tea bud detection and picking sequence planning are two challenges that hinder the development of these robots. To this end, two lightweight neural networks are investigated to tackle these two problems. A state-of-the-art detection network YOLOX-S is first deployed to quickly identify tea buds. Second, the picking sequence planning of the detected tea buds is formulated as a traveling salesman problem (TSP). Then a modified pointer network is proposed to solve the TSP, improving the pointer network by replacing its recurrent neural network with a simple self-attention layer and optimizing its parameters using a reinforcement learning algorithm. The experimental results show that YOLOX-S achieves an average precision of 0.642 and an average running time of 17.43 ms; for the problem of up to 100 tea buds in the two-dimensional interval [0, 1] × [0, 1], the modified pointer network solves the TSP with an average path length of 8.30 and an average running time of 1.69 ms. These results demonstrate that YOLOX-S and the modified pointer network can efficiently solve the tea bud detection and picking sequence planning problem without losing too much accuracy, which provides technical support for real-time tea-harvesting robots.