Cross-modal visual tracking method and device based on adaptive convolution

The invention discloses a cross-modal visual tracking method and device based on self-adaptive convolution, and belongs to the technical field of computer vision, and the method comprises the steps: inputting a pair of registered multi-modal images, generating a weight tensor corresponding to the size of a feature graph after each layer of convolution through a self-adaptive convolution module, and generating a weight tensor corresponding to the size of a feature graph; self-adaptive fusion is carried out on input among different modals pixel by pixel, self-adaptive fusion of two modal features is carried out again on a fusion result and a single input modal feature, and cross-modal information interaction and single modal information enhancement are achieved; fine-tuning the fully connected layer according to a first frame collection sample of each video to cope with an instance-specific challenge; and finally, sending to the last layer of the full connection layer for binary classification operation to obta