Reciprocal Teacher-Student Learning via Forward and Feedback Knowledge Distillation

Knowledge distillation (KD) is a prevalent model compression technique in deep learning, aiming to leverage knowledge from a large teacher model to enhance the training of a smaller student model. It has found success in deploying compact deep models in intelligent applications like intelligent tran...

Ausführliche Beschreibung