Concept Propagation Based on Visual Similarity: Application to Medical Image Annotation

This paper presents an approach for image annotation propagation to images which have no annotations. In some specific domains, the assumption that visual similarity implies (partial) semantic similarity can be made. For instance, in medical imaging, two images of the same anatomic part in a given modality have a very similar appearance. In the proposed approach, a conceptual indexing phase extracts concepts from texts; a visual similarity between images is computed and then combined with conceptual text indexing. Annotation propagation driven by prior knowledge on the domain is finally performed. Domain knowledge used is a meta-thesaurus for both indexing and annotations propagation. The proposed approach has been applied on the imageCLEF medical image collection.