Multi-modal skeleton action recognition method and device

The invention relates to the field of artificial intelligence visual language multi-modality, in particular to a multi-modality skeleton action recognition method and device. The method comprises the following steps: acquiring and preprocessing skeleton data, and extracting corresponding skeleton original features; a visual encoder and a multi-layer perceptron are used for extracting corresponding visual features from the preprocessed skeleton data, meanwhile, a text prompt mapper is used for mapping action labels into texts, and a text encoder and the multi-layer perceptron are used for extracting corresponding language features from the texts; a loss value is calculated through a loss function, a visual encoder, a text decoder and a multi-layer perceptron are trained, and the loss function is composed of a visual loss function, a visual language loss function and a language decoding loss function; and testing by using the trained visual encoder to obtain a skeleton action recognition and classification resu