Vision-based extraction of spatial information in grape clusters for harvesting robots

Grapes are likely to have collisions and be damaged by manipulations when harvesting grape clusters. To conduct an undamaged robotic harvesting, this paper focuses mainly on locating the spatial coordinates of the cutting points on a peduncle of grape clusters for the end-effector and determining the bounding volume of the grape clusters for the motion planner of the manipulator. A method for acquiring spatial information from grape clusters is presented based on binocular stereo vision. This method includes four steps: (1) calibrating the binocular cameras and rectifying the images, (2) detecting the cutting points on the peduncle and the centres of the grape berries, (3) extracting three-dimensional spatial coordinates of the points detected in step 2, and (4) calculating the bounding volume of the grape clusters. A total of 300 images were captured in the vineyard and were tested to validate the method for the cutting point detection, and the success rate was approximately 87%. The accuracy of the localisation of the cutting points was determined under outdoor conditions, and the accuracy in the Z and X directions was 12 mm and 9 mm, respectively. The acquired bounding volume of the grape cluster was compared with manual measurements, and errors in the height and maximum diameter were less than 17 mm and 19 mm, respectively. The elapsed time of the whole algorithm was less than 0.7 s. The demonstrated performance of this developed method indicated that it could be used on harvesting robots. •A method for detecting and locating the cutting point on a grape peduncle was developed.•The bounding volume of the grape cluster was calculated to plan a free-collision path.•The demonstrated performance indicated that it could be used on harvesting robots.