Rollout strategy for Hidden Markov Model (HMM)-based dynamic sensor scheduling

In this paper, a hidden Markov model (HMM)-based dynamic sensor scheduling problem is formulated, and solved using rollout concepts to overcome the computational intractability of the dynamic programming (DP) recursion. The problem considered here involves dynamically sequencing a set of sensors to minimize the sum of sensor cost and the HMM state estimation error cost. The surveillance task is modeled as a single HMM with multiple emission matrices corresponding to each of the sensors. The rollout information gain (RIG) algorithm proposed herein employs the information gain (IG) heuristic as the base algorithm. The RIG algorithm is illustrated on an intelligence, surveillance, and reconnaissance (ISR) scenario of a village for the presence of weapons and terrorists/refugees. Extension of the RIG strategy to monitor multiple HMMs involves combining the information gain heuristic with the auction algorithm that computes the K -best assignments at each decision epoch of rollout.