Multi-View correlation distillation for incremental object detection

•A novel incremental object detection method is proposed to explore and transfer the multi-view correlations in the feature space of the object detector.•The correlation distillation losses for the sample-specific selective features from three views (channel-wise, point-wise and instance-wise) are designed.•A new metric called Stability-Plasticity-mAP (SPmAP) is proposed to measure the incremental object detector performance.•The proposed method achieves competitive results compared with previous incremental object detection methods. In real applications, new object classes often emerge after the detection model has been trained on a prepared dataset with fixed classes. Fine-tuning the old model with only new data will lead to a well-known phenomenon of catastrophic forgetting, which severely degrades the performance of modern object detectors. Due to the storage burden, data privacy and time consumption, sometimes it is impractical to train the model from scratch with all data of both old and new classes. In this paper, we propose a novel Multi-View Correlation Distillation (MVCD) based incremental object detection method, which explores the intra-feature correlations in the feature space of the object detector. To better transfer the knowledge learned from the old classes and maintain the ability to learn new classes, we select the sample-specific discriminative features from channel-wise, point-wise and instance-wise views. Meanwhile, the correlation distillation losses on the selective features are designed to regularize the learning of the incremental object detector. A new metric named Stability-Plasticity-mAP (SPmAP) is proposed to evaluate the incremental learning performance as a complementary metric to mAP, which integrates the metrics for the stability on old classes and the plasticity on new classes in incremental object detection. The extensive experiments conducted on VOC2007 and COCO demonstrate that MVCD achieves a better trade-off between stability and plasticity than state-of-the-art first-order distillation-based incremental object detection methods.