Person authentication by fusing face and speech information

A person authentication technique using two modalities is presented. Results from individual experts, namely face and speech recognisers, are merged by a supervisor. The visual part involves matching of a coarse grid containing Gabor phase information from face images. The acoustic part is performed by a text-dependent speaker verification system based on Hidden Markov Models, which assumes as text a spelled sequence of digits. The merging of individual decisions is accomplished by one of two different methods: a simple averaging and a more sophisticated Bayesian method. Experimental results show that even the simple method provides improvements compared to single modalities. The improvements are significant with the Bayesian method. These results show that the use of two modalities increases authentication performance at least under certain circumstances.