Siamese transformer network-based similarity metric learning for cross-source remote sensing image retrieval

As a fundamental technique for mining and analysis of remote sensing (RS) big data, content-based remote sensing image retrieval (CBRSIR) has received a lot of attention. Recently, cross-source CBRSIR (CS-CBRSIR) has become one of the most challenging tasks in the RS community. Due to the data drift issue, it is hard to find a proper similarity metric function to accurately measure similarities between the RS images from different sources. To address this issue, instead of directly using the manually designed similarity metrics, we propose an end-to-end similarity metric learning network, i.e., Siamese Transformer Network (STN) for CS-CBRSIR. Specifically, the proposed STN consists of three modules: (1) feature extraction module, which is a network combining Vision Transformer (ViT) with convolution layers, named as ConViT, (2) similarity metric function, which is a fully connected neural network (FCNN) aiming to compute the similarity between the output features from different sources, and (3) smooth average-precision (Smooth-AP) loss function, which measures the surrogate loss of standard AP metric to optimize the similarity metric function through backpropagation. Afterward, the learned similarity metric function can be adopted to implement the CS-CBRSIR accurately. Extensive experiments and ablation studies demonstrate that the proposed approach achieves promising performance in the CS-CBRSIR task.