Multi-frequency PolSAR Image Fusion Classification Based on Semantic Interactive Information and Topological Structure

Compared with the rapid development of single-frequency polarimetric SAR (PolSAR) image classification technology, there is less research on the land cover classification of multi-frequency PolSAR (MF-PolSAR) images. And the deep learning methods among them are mainly based on convolutional neural networks (CNNs), only local spatiality is considered but the nonlocal relationship is ignored. Therefore, this paper proposes the MF semantics and topology fusion (MF-STF) model based on semantic interaction and nonlocal topological structure to improve MF-PolSAR classification performance. During MF-STF optimization, the semantic information-based classification (SIC) and topological property-based classification (TPC) work collaboratively, not only fully leveraging the complementarity of bands, but also combining local and nonlocal spatial information to improve the discrimination of different categories. For SIC, the designed cross-band interactive feature extraction (CIFE) module is embedded to explicitly model the deep semantic correlation among bands, thereby leveraging the complementarity of bands to make ground objects more separable. In TPC, the graph sample and aggregate network (GraphSAGE) is employed to dynamically capture the representation of nonlocal topological relations between land cover categories. In this way, the robustness of classification can be further improved by combining nonlocal spatial information. Finally, a MF weighted fusion (MFWF) strategy is proposed to merge inference from different bands, so as to make the MF joint classification decisions of SIC and TPC. Notably, its weights are adjusted based on the total model loss. The effectiveness of the proposed modules is proved by ablation experiments on three measured MF-PolSAR datasets. In addition, the comparative experiments show that MF-STF can achieve more competitive classification performance than some state-of-the-art methods.