Speech classification using penalized logistic regression with hidden Markov model log-likelihood regressors

Penalized logistic regression (PLR) is a well-founded discriminative classifier with long roots in the history of statistics. Speech classification with PLR is possible with an appropriate choice of map from the space of feature vector sequences into the Euclidean space. In this talk, one such map is presented, namely, the one that maps into vectors consisting of log-likelihoods computed from a set of hidden Markov models (HMMs). The use of this map in PLR leads to a powerful discriminative classifier that naturally handles the sequential data arising in speech classification. In the training phase, the HMM parameters and the regression parameters are jointly estimated by maximizing a penalized likelihood. The proposed approach is shown to be a generalization of conditional maximum likelihood (CML) and maximum mutual information (MMI) estimation for speech classification, leading to more flexible decision boundaries and higher classification accuracy. The posterior probabilities resulting from classification with PLR allow for continuous speech recognition via N-best or lattice rescoring.