Evidential split-and-merge: Application to object-based image analysis

This paper addresses the difficult problem of segmenting objects in a scene and simultaneously estimating their material class. Focusing on the case where, individually, no dataset can achieve such a task, multiple sensor datasets are considered, including some images for retrieving the spatial information. The proposed approach is based on mutual validation between class decision (using the most relevant dataset) and segmentation (derived from image data). The main originality lies in the ability to make these two modules (classification and segmentation) interactive. Specifically, our application focuses on object-level material labeling using classic RGB images, laser profilometer images and a NIR spectral sensor. Starting from a superpixel segmentation, the relevant data are introduced as constraints modifying the initial segmentation in a split-and-merge process, which interacts with the material labeling process. In this work, we use the belief function framework to model the information extracted from each kind of data and to transfer it from one processing module to another. In particular we show the relevance of evidential conflict measure to drive the split process and to control the merge one. Experiments have been performed on actual scenes with stacked objects and difficult cases of material such as transparent polymers. They allow us to assess the performance of the proposed approach both in terms of material labeling and object segmentation as well as to illustrate some borderline cases.