Visual Workflow Recognition Using a Variational Bayesian Treatment of Multistream Fused Hidden Markov Models

In this paper, we provide a variational Bayesian (VB) treatment of multistream fused hidden Markov models (MFHMMs), and apply it in the context of active learning-based visual workflow recognition (WR). Contrary to training methods yielding point estimates, such as maximum likelihood or maximum a posteriori training, the VB approach provides an estimate of the posterior distribution over the MFHMM parameters. As a result, our approach provides an elegant solution toward the amelioration of the overfitting issues of point estimate-based methods. Additionally, it provides a measure of confidence in the accuracy of the learned model, thus allowing for the easy and cost-effective utilization of active learning in the context of MFHMMs. Two alternative active learning algorithms are considered in this paper: query by committee, which selects unlabeled data that minimize the classification variance, and a maximum information gain method that aims to maximize the alteration in model variance by proper data labeling. We demonstrate the efficacy of the proposed treatment of MFHMMs by examining two challenging WR scenarios, and show that the application of active learning, which is facilitated by our VB approach, allows for a significant reduction of the MFHMM training costs.