A hierarchical segmentation algorithm for face analysis. Application to lipreading

A hierarchical algorithm for face analysis is presented. A color video sequence of the speaker's face is acquired under natural lighting conditions and without any particular make-up. The application aims at providing geometrical features of the face for scalable video transmission when no specific model of the speaker face is assumed. First, a logarithmic hue transform is performed from RGB to HI (hue, intensity) color space. Next, a Markov random field modeling regularizes motion and hue information within a spatiotemporal neighborhood. The hierarchical segmentation labels the different areas of the face. Results are shown on the lower part of the face and compared with a standard color segmentation algorithm (fuzzy c-means). A speaker's lip shape with inner and outer borders is extracted from the final labeling and used to initialize an active contour stage.