A performance evaluation of gradient field HOG descriptor for sketch based image retrieval

► Comprehensive evaluation of new descriptor GF-HOG for Sketch Based Image Retrieval (SBIR). ► Compares accuracy, speed and affine invariance to six state of the art SBIR descriptors using several distance measures. ► New FlickR source annotated image dataset for SBIR. ► Fuses GF-HOG (shape based) r...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Computer vision and image understanding 2013-07, Vol.117 (7), p.790-806
Hauptverfasser: Hu, Rui, Collomosse, John
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:► Comprehensive evaluation of new descriptor GF-HOG for Sketch Based Image Retrieval (SBIR). ► Compares accuracy, speed and affine invariance to six state of the art SBIR descriptors using several distance measures. ► New FlickR source annotated image dataset for SBIR. ► Fuses GF-HOG (shape based) retrieval with text keywords for semantic SBIR. ► All source code and data to be released upon publication. We present an image retrieval system for the interactive search of photo collections using free-hand sketches depicting shape. We describe Gradient Field HOG (GF-HOG); an adapted form of the HOG descriptor suitable for Sketch Based Image Retrieval (SBIR). We incorporate GF-HOG into a Bag of Visual Words (BoVW) retrieval framework, and demonstrate how this combination may be harnessed both for robust SBIR, and for localizing sketched objects within an image. We evaluate over a large Flickr sourced dataset comprising 33 shape categories, using queries from 10 non-expert sketchers. We compare GF-HOG against state-of-the-art descriptors with common distance measures and language models for image retrieval, and explore how affine deformation of the sketch impacts search performance. GF-HOG is shown to consistently outperform retrieval versus SIFT, multi-resolution HOG, Self Similarity, Shape Context and Structure Tensor. Further, we incorporate semantic keywords into our GF-HOG system to enable the use of annotated sketches for image search. A novel graph-based measure of semantic similarity is proposed and two applications explored: semantic sketch based image retrieval and a semantic photo montage.
ISSN:1077-3142
1090-235X
DOI:10.1016/j.cviu.2013.02.005