Adaptive Knowledge Distillation for Lightweight Remote Sensing Object Detectors Optimizing

Lightweight object detector is currently gaining more and more popularity in remote sensing. In general, it is hard for lightweight detectors to achieve competitive performance compared with the traditional deep models, while knowledge distillation (KD) is a promising training method to tackle the issue. Since the background is more complicated and the object size varies extremely in remote sensing images, it will deliver lots of noise and affect the training performance when directly applying the existing KD methods. To tackle the above problems, we propose an adaptive reinforcement supervision distillation (ARSD) framework to promote the detection capability of the lightweight model. First, we put forward a multiscale core features imitation (MCFI) module for transferring the knowledge of features, which can adaptively select the multiscale core features of objects for distillation and focus more on the features of small objects by an area-weighted strategy. In addition, a strict supervision regression distillation (SSRD) module is designed to select the optimal regression results for distillation, which facilitates the student to effectively imitate the more precise regression output of the teacher network. Massive experiments on a large-scale Dataset for Object deTection in Aerial images (DOTA), object DetectIon in Optical Remote sensing images (DIOR), and Northwestern Polytechnical University Very-High-Resolution 10-class (NWPU VHR) datasets prove that ARSD outperforms the existing distillation state-of-the-art (SOTA) methods. Moreover, the performance of the lightweight model trained with our method transcends other classic heavy and lightweight detectors, which beneficiates the development of lightweight models.