Bag of spatio-visual words for context inference in scene classification

In the “bag of visual words (BoVW)” representation each image is represented by an unordered set of visual words. In this paper, a novel approach to encode ordered spatial configurations of visual words in order to add context in the representation is presented. The proposed method introduces a bag of spatio-visual words representation (BoSVW) obtained by clustering of visual words' correlogram ensembles. Specifically, the spherical K-means clustering algorithm is employed accounting for the large dimensionality and the sparsity of the proposed spatio-visual descriptors. Experimental results on four standard datasets show that the proposed method significantly improves a state-of-the-art BoVW model and compares favorably to existing context-based scene classification approaches. ► Reform BoVw representation to include spatio-contextual information. ► Spherical k-means for high-dimentional spatio-visual data clustering. ► Improves a state-of-the-art BoVw model on 4 reference datasets. ► Compares favorably to existing context-based scene classification approaches.