End-to-end learning for weakly supervised video anomaly detection using Absorbing Markov Chain

We propose a principled deep neural network framework with Absorbing Markov Chain (AMC) for weakly supervised anomaly detection in surveillance videos. Our model consists of both a weakly supervised binary classification network and a Graph Convolutional Network (GCN), which are jointly optimized by backpropagation. Unlike the previous works that employ AMC for label noise filtering in a post-processing step, the proposed framework migrates the component inside the GCN part of our model and realizes an end-to-end learning network. The integration of the AMC module into our deep neural network model enables us to learn the associated parameters automatically, which is helpful to improve the quality of segment-wise label estimation via tightly-coupled processing between the main network and the AMC module. In addition, we introduce a pseudo-labeling strategy based on Gaussian mixture model to fully utilize examples in abnormal videos. Our algorithm achieves outstanding performance compared to the state-of-the-art weakly supervised anomaly detection methods on UCF-Crime and ShanghaiTech datasets. •We introduce a novel approach for weakly supervised video anomaly detection.•We adopt the concept of Absorbing Markov Chain to alleviate noisy predictions.•Our model integrates AMC and Graph Convolutional Network seamlessly.•We achieve outstanding performance on the UCF-Crime and ShanghaiTech datasets.