Cross-layer classification framework for automatic social behavioural analysis in surveillance scenario

The increasing demand for human activity analysis in surveillance scenarios has been triggered by the emergence of new features and concepts to help in identifying activities of interest. However, the characterisation of individual and group behaviours is a topic not so well studied in the video surveillance community due to not only its intrinsic difficulty and large variety of topics involved, but also because of the lack of valid semantic concepts that relate human activity to social context. In this paper, we address the topic of social semantic meaning in a well-defined surveillance scenario, namely shopping mall, and propose new definitions of individual and group behaviour that consider environment context, a relational descriptor that emphasises position and attention-based characteristics, and a new classification approach based on mini-batches. We also present a wide evaluation process that analyses the sociological meaning of the individual features and outlines the performance impact of automatic features extraction processes into our classification framework. We verify the discriminative value of the selected features, state the descriptor performance and robustness over different stress conditions, confirm the advantage of the proposed mini-batch classification approach which obtains promising results, and outline future research lines to improve our novel social behavioural analysis framework.