Online Learning of Minmax Solutions for Distributed Estimation and Tracking Control of Sensor Networks in Graphical Games

In this article, a new target tracking algorithm that is based on a distributed estimation-based control protocol is developed for multiple moving sensors subject to adversarial inputs. An augmented system composed of both the estimation error dynamics and the tracking error dynamics is introduced for the distributed estimation and target tracking control problem. We integrate estimation and control to simultaneously minimize infinite-horizon estimation and tracking errors in sensor networks with disturbances. A Minmax strategy as an alternative goal to Nash equilibrium is proposed to compute the sensor's optimal motion controls in the presence of worst-case neighbor controls and disturbances. In contrast to Nash, Minmax yields a distributed algebraic Riccati equation that is easily solved locally by each sensor. A method based on reinforcement learning is given to compute sensor motion inputs so that all sensors estimate and track the target states. Value function approximation using neural networks is used to solve the distributed Bellman equations online based on the real-time observed data. Proofs are given to guarantee the performance of the combined distributed estimation and tracking algorithms. Finally, the simulation examples show the effectiveness of the proposed algorithm.