Diversity Knowledge Distillation for LiDAR-based 3D Object Detection

The light detection and ranging (LiDAR) sensor enables high-quality 3D object detection, which is critical in autonomous driving applications. However, accurate detectors require more computing resources owing to the discreteness and disorder of point cloud data. To resolve this problem, we propose diversity knowledge distillation or 3D object detection, which distills the knowledge from a two-stage high-accuracy detector to a faster one-stage detector. This framework includes methods to match the bounding box predictions of one-stage student and two-stage teacher detectors with inconsistent numbers. Accordingly, we design a response-based distillation method to perform the distillation. Then, a diversity feature score is proposed to guide the student in selecting the parts that need more attention on the middle-layer feature map and the region of interest output by the distillation process. Experiments demonstrate that the proposed method can enhance the performance of a one-stage detector without increasing the computation of the mode in the test stage.