Object sorting method and arm-hand system based on multimode perception and hardness tactile detection

The invention discloses an object sorting method and arm-hand system based on multimode perception and hardness touch detection, and the method comprises the steps: 1, constructing a training set of a fruit hardness classification network, and training the fruit hardness classification network; 2, performing hand-eye calibration on the depth camera, and mapping a target position visually perceived by the depth camera to a motion trail of the mechanical arm; 3, the YOLOv8 model is trained; 4, under the joint assistance of the depth camera and the trained YOLOv8 model, a mechanical arm grabs a target fruit through a mechanical clamping jaw, tactile information of a first flexible tactile sensor is transmitted to the trained fruit hardness classification network, and the hardness of the target fruit is obtained; and 5, the maturity of the fruits is classified, and the fruits are sorted to designated positions. According to the full-tactile ripeness detection method, the full-tactile ripeness detection of the tar