Lip reading using optical flow and support vector machines

This paper presents a lip reading technique to classify the discrete utterances without evaluating the acoustic signals. The reported technique analysis the video data of lip motions by computing the optical flow (OF). The statistical properties of the vertical OF component were used to form the feature vectors for training the support vector machines (SVM) classifier. The impact of the variation in speed/velocity of speaking on the performance of the system was minimized by removing the zero energy frames and normalizing the number of frames by interpolation. The resulting system is an efficient visual viseme classifier with high accuracy (95.9%), specificity (98.1%) and sensitivity (66.4%). The results of the experiments demonstrate the developed technique is insensitive to inter speaker variations.