Composed complex-cue histograms: An investigation of the information content in receptive field based image descriptors for object recognition

► New receptive-field histograms based on catalogue of scale-space operations. ► Gaussian derivatives and differential invariants from color-opponent channels. ► Composed histograms give better performance than primitive image features. ► Capture the co-variation of different primitive image cues. ► Sparse representation captures higher-dimensional histograms efficiently. Recent work has shown that effective methods for recognizing objects and spatio-temporal events can be constructed based on histograms of receptive field like image operations. This paper presents the results of an extensive study of the performance of different types of receptive field like image descriptors for histogram-based object recognition, based on different combinations of image cues in terms of Gaussian derivatives or differential invariants applied to either intensity information, color-opponent channels or both. A rich set of composed complex-cue image descriptors is introduced and evaluated with respect to the problems of (i) recognizing previously seen object instances from previously unseen views, and (ii) classifying previously unseen objects into visual categories. It is shown that there exist novel histogram descriptors with significantly better recognition performance compared to previously used histogram features within the same class. Specifically, the experiments show that it is possible to obtain more discriminative features by combining lower-dimensional scale-space features into composed complex-cue histograms. Furthermore, different types of image descriptors have different relative advantages with respect to the problems of object instance recognition vs. object category classification. These conclusions are obtained from extensive evaluations on two mutually independent data sets. For the task of recognizing specific object instances, combined histograms of spatial and spatio-chromatic derivatives are highly discriminative, and several image descriptors in terms rotationally invariant (intensity and spatio-chromatic) differential invariants up to order two lead to very high recognition rates. For category classification, primary information is contained in both first-and second-order derivatives, where second-order partial derivatives constitute the most discriminative cue. Dimensionality reduction by principal component analysis and variance normalization prior to training and recognition can in many cases lead to a significant increase in recognition or clas