Contour-driven Sumi-e rendering of real photos

We propose a sketch-based system for rendering oriental brush strokes on complex shapes. While previous research has focused on methods for converting user-specified trajectories into oriental ink painting (Sumi-e) strokes, we propose an approach that takes as input the contours of complex shapes, estimates automatically the sizes of the brush footprints, their orientations, and the brush trajectory. Then, it renders them into oriental ink paintings. The approach performs in three steps; first complex shapes are decomposed into elementary shapes that can be rendered with a single brush stroke. Second, the brush trajectory estimation is formulated as the minimization of an energy function that measures the quality of the trajectory. Finally, the estimated trajectories are rendered into brush strokes by mapping on them footprint textures scanned from real images. We demonstrate that the proposed contour-driven approach is particularly suitable for converting real images into Sumi-e paintings with minimum interaction. To this end we develop a system where images are either interactively or automatically segmented into elementary shapes and converted into Sumi-e paintings. For the automatic segmentation, we observe that the strokes in Sumi-e paintings are nearly parallel to the edge tangent flow (ETF). We propose a segmentation procedure that is based on clustering the curves obtained by integrating the ETF of real images. Experiments on complex shapes show that the proposed contour-based approach produces a large variety of complex strokes compared to trajectory-based approaches. Overview of the painting process: real images are segmented interactively or automatically into shape contours. Complex contours are then decomposed into elementary shapes that can be rendered with a single stroke. The elementary strokes are independently processed: the optimal brush trajectory is estimated and Sumi-e strokes are generated. Finally, the strokes are synthesized to generate a full painting. [Display omitted] ► This article: ► Develops methods for converting interactively real images into Sumi-e paintings. ► Proposes an extension for automating the process. The distinct feature of our approach is the ways strokes are generated: ► Instead of using the medial axis as the brush trajectory, we automatically estimate the optimal trajectory, and the optimal size and orientation of each footprint, given the contour of the shape to paint. ► We formulate the problem as the minim