Sub-scene segmentation using constraints based on Gestalt principles

•Sub-scenes are more integrated and semantically consistent regions.•Proximity grouping is formulated more appropriately using influence areas.•Optimal segmentation is achieved by a self-determined optimal retrieval strategy.•The method ignores unimportant details and results in a more integrated segmentation. In this paper, an unsupervised sub-scene segmentation method is proposed. It emphasizes on generating more integrated and semantically consistent regions instead of homogeneous but detailed over-segmented regions usually produced by conventional segmentation methods. Several properties of sub-scenes are explored such as proximity grouping, area of influence, similarity and harmony based on psychological principles. These properties are formulated into constraints that are used directly in the proposed sub-scene segmentation. A self-determined approach is conducted to get the optimal segmentation result based on the characteristics of each image in an unsupervised manner. The proposed method is evaluated over three datasets. For quantitative evaluation, the performance of the proposed method is on par with state-of-the-art unsupervised segmentation methods; for qualitative evaluation, the proposed method handles various sub-scenes well, and produces neater results. The sub-scenes segmented by the proposed method are generally consistent with natural scene categories.