MTSCD-Net: A network based on multi-task learning for semantic change detection of bitemporal remote sensing images

•The feature aggregation module is designed for the features fusion.•The change information extraction module is designed to obtain robust features.•The correlation between subtasks is considered during the decoding phase. In recent years, change detection has been one of the hot research topics within the field of remote sensing. Previous studies have concentrated on binary change detection (BCD), but it doesn’t meet the current needs. Therefore, semantic change detection (SCD) is also gradually developing, which focuses on determining the specific changed type while obtaining changed areas. In the paper, we propose a multi-task learning method (MTSCD-Net) for SCD task. The SCD task is decoupled into two related subtasks, semantic segmentation (SS) and BCD, then unifies them under the same framework. Multi-scale features are extracted using the Siamese semantic-aware encoder based on Swin Transformer, and the aggregation module is designed to combine features. Then, the change information extraction module is designed to enhance the capacity to express features by fully integrating the two-level difference features that are generated from fused features. Moreover, in the decoder stage, the spatial attention weight map is obtained using the features of the BCD subtask, which provides location prior information for the features of the SS subtask. It helps fully explore the correlation between the two subtasks. The two loss functions of subtasks are weighted to train MTSCD-Net. The comparative experiments results on two typical SCD datasets confirm the advantage of MTSCD-Net for SCD task. For the SeK index, MTSCD-Net achieves 3.96% and 20.57% on HRSCD and SECOND datasets, respectively. This outperforms other comparative methods such as Bi-SRNet (which achieves 4.86% and 1.47% higher on two datasets, respectively). The same is true for the Score metric. Moreover, the ablation experiment results confirm the effectiveness of key modules.