Robot classification detection method and system based on multi-mode multi-task learning

The invention relates to the technical field of robots, in particular to a robot classification detection method and system based on multi-modal multi-task learning, and the method comprises the steps: S1, constructing a multi-modal data set, and carrying out the data preprocessing; s2, aligning the semantic information data set and the image data set; s3, constructing a multi-modal target detection model, inputting a multi-modal data set into the model to perform multi-task learning, performing feature extraction on the multi-modal data set, performing feature fusion on extracted visual image features and semantic information features, and calculating a weighted sum of the visual image features of the robot by using a core semantic attention mechanism to obtain a target detection result; and training the model by optimizing the weighted sum. According to the robot classification detection method based on multi-modal multi-task learning provided by the invention, the image and semantic information of the robo