Learning to match features with discriminative sparse graph neural network

We propose a cluster-based sparse graph network to improve the efficiency of image feature matching. This architecture clusters keypoints with high correlations into the same subgraphs, where each keypoint interacts only with others within the same subgraph. This strategy effectively reduces the spread of redundant messages and boosts the efficiency of message transmission. A unique coarse-to-fine paradigm is proposed for the incremental construction of sparse graphs, facilitating the evolution of subgraphs from coarse to fine, which enhances keypoint correlation and reduces misclassification. Additionally, the introduction of global tokens within each subgraph enables the learning of global information through interactions with a limited number of global tokens, further minimizing the impact of misclassification by broadening the scope of learning beyond the limits of individual subgraphs. The methodology demonstrates competitive performance in a range of vision tasks, including pose estimation, visual localization, and homography estimation. Compared to complete graph networks, it reduces time and memory consumption by 91% and 46%, respectively, during dense matching. Moreover, building on this foundational architecture, we introduce a novel hierarchical approach for visual localization, utilizing a two-stage sparse-to-dense matching process, achieves a substantial 31.8% decrease in time consumption while maintains competitive accuracy. •A cluster-based sparse graph network architecture is introduced to enhance feature matching efficiency by clustering keypoints with strong correlations into the same subgraph, which reduces the propagation of redundant message, leading to more efficient message transmission.•A coarse-to-fine approach is adopted for the progressive construction of sparse graphs, leading to subgraphs that evolve from coarse to fine, effectively enhancing keypoint correlation while reducing subgraph size.•Each subgraph generates a global token, allowing interactions with few global tokens to learn global insights. This reduces misclassifications by expanding learning beyond just the subgraph’s confines.•The proposed method has achieved state-of-the-art results in various vision tasks, including pose estimation, visual localization, and homography estimation. It significantly reduces time consumption and memory usage by 91% and 46%, respectively, in dense detection compared to the fully connected graph feature-matching method.•Building on th