Detecting visual anomalies in an industrial environment: Unsupervised methods put to the test on the AutoVI dataset

The methods for unsupervised visual inspection use algorithms that are developed, trained and evaluated on publicly available datasets. However, these datasets do not reflect genuine industrial conditions, and thus current methods are not evaluated in real-world industrial production contexts. To answer this shortcoming, we introduce AutoVI, an industrial dataset of visual defects that can be encountered on automotive assembly lines. This dataset, comprising six inspection tasks, was designed as a benchmark to assess the performance of defect detection methods under realistic acquisition conditions. We analyze the performance of current state-of-the-art methods and discuss the difficulties specifically encountered in the industrial context. Our results show that current methods leave considerable room for improvement. We make AutoVI publicly available to develop unsupervised detection methods that will be better suited to real industrial tasks. •Review of existing public datasets and highlight of the need for a genuine dataset.•Introduction of AutoVI, the first genuine industrial dataset for anomaly detection.•Benchmark study of current state-of-the-art anomaly detection methods on AutoVI.•Current methods are not able to handle genuine industrial environments.•AutoVI aims to help the community improve unsupervised defect detection models.