PW-MFL: Promoting Semantic Segmentation in Resolution-Degraded Aerial Images via Pixel-Wise Mutual-Feed Learning

Due to variable imaging conditions, resolution degradation often occurs in aerial images, which in turn impairs the performance upper bound of semantic segmentation (SS). To solve this problem, super-resolution (SR) is placed before SS as a preprocessing subtask in most existing methods. The above two subtasks often form a unidirectional open-loop structure for relatively independent optimization, which constrains the ultimate segmentation performance improvement. To break down information barriers among them and form a more compact overall optimization, we propose an effective learning method named as pixel-wise mutual-feed learning (PW-MFL) for segmenting images with resolution degradation. The key is to build auxiliary bidirectional connections, which contribute to the mutual pixel-wise spatial and feature information guidance during training. The feed-forward connection is realized by the self-attention context correlation (SACC) module, which enhances the intraclass semantic features of pixel positions with poor SR performance by the fusion of that with superior performance. The feedback connection is achieved by the semantic weighted mapping (SWM) module, which aims to activate and adjust the detailed features of SR in incorrectly segmented pixel positions via the semantic feature information. In addition, the pixel-aware optimization (PAO) strategy is proposed to give more attention to optimizing specific pixel positions based on spatial information. Extensive experiments are conducted on three representative remote sensing segmentation benchmarks, ISPRS Vaihingen, ISPRS Potsdam, and iSAID datasets. The state-of-the-art (SOTA) segmentation level in resolution-degraded aerial images is achieved through the proposed learning method.