Unsupervised lip segmentation based on quad-tree MRF framework in wavelet domain

•Lip reading is a novel and intelligent way of human-computer interaction.•Quad-tree Markov random field based unsupervised lip segmentation is proposed.•Multi-scale characteristics of wavelet transform is applied to MRF segmentation.•Better lip segmentation accuracy is achieved compared to other MRF-based methods. In this paper, we propose a new extended method to implement lip segmentation. Based on a quad-tree structure (QTS) Markov random field (MRF) model under unsupervised situation, the proposed method achieves the lip segmentation in wavelet domain. Firstly, we set up a multi-layer hierarchical model, in which each pixel of every layer corresponds to the four nodes in quad-tree structure. Then the probability of a branch node can be calculated by using the probability of the previous branch node through the tree structure easily. Subsequently, a Markov random field derived from the model is obtained so that the segmentation problem is formulated as a labeling optimization problem in the framework of the maximum a posteriori Markov random field (MAP-MRF). Assuming that the pre-assigned cluster of data segments may overestimate the underlying fact, and leads to over-segmentation, we propose a variable-weight segmentation approach to improve the robustness of the segmentation. The experimental results show that this method has better segmentation accuracy than traditional methods.