A multi-criteria context-sensitive approach for social image collection summarization

Recent increase in the number of digital photos in the content sharing and social networking websites has created an endless demand for techniques to analyze, navigate, and summarize these images. In this paper, we focus on image collection summarization. Earlier methods in image collection summarization consider representativeness and diversity criteria while recent ones also consider other criteria such as image quality, aesthetic or appeal. In this paper, we propose a multi-criteria context-sensitive approach for social image collection summarization. In the proposed method, two different sets of features are combined while each one looks at different criteria for image collection summarization: social attractiveness features and semantic features. The first feature set considers different aspects that make an image appealing such as image quality, aesthetic, and emotion to create attractiveness score for input images while the second one covers semantic content of images and assigns semantic score to them. We use social network infrastructure to identify attractiveness features and domain ontology for extracting ontology features. The final summarization is provided by integrating the attractiveness and semantic features of input images. The experimental results on a collection of human generated summaries on a set of Flickr images demonstrate the effectiveness of the proposed image collection summarization approach.