Extensive Knowledge Distillation Model: An End-to-End Effective Anomaly Detection Model for Real-Time Industrial Applications

Detecting anomalies is an essential task in many industries. Current state-of-the-art methods rely on a large number of parameters for high accuracy, which may not be suitable for implementing cost-effective real-time applications. Additionally, developing robust detection models is difficult due to limited anomaly samples. To address these issues, we propose an end-to-end anomaly detection method that utilizes effective data generation and comprehensive knowledge distillation. In particular, the proposed approach first employs a highly effective generative model to generate realistic anomaly images. It then transfers the pretrained master network's essential intermediate layers and final layer knowledge to a novice network by using the knowledge distillation technique. In the conducted experiments with 4 real-life datasets, the proposed model outperforms its counterparts, including state-of-the-art models, by 0.6% on MNIST and CIFAR-10 datasets, 0.2% on the custom dataset, and stays competitive on the MVTec AD dataset. Additionally, the proposed model outperforms all of its peers in trainable parameter numbers by having only 0.17 M parameters. This is at least twice as few parameters as the baseline model. Overall, the proposed approach offers an efficient solution to anomaly detection that achieves high accuracy despite limited anomaly samples and fewer parameters.