Robust output group formation tracking control of heterogeneous multi-agent systems with multiple leaders using reinforcement learning

This paper studies the distributed output formation tracking problem of grouped heterogeneous multi-agent systems under multiple leaders and uncertainties using reinforcement learning (RL). The outputs of followers are supposed to achieve robust tracking to the respective convex point of group leaders while generating an expected time-varying formation configuration. First, a distributed adaptive observer is designed under a directed graph to coordinate the multiple group leaders while estimating the leaders’ dynamics in finite-time. The adaptive mechanism avoids global information of the graph. Second, an optimal tracking problem with respect to the observer is formulated for each follower, while the feedback tracking controller is derived using an action-dependent RL algorithm. An extended learning process for essential dynamics is constructed using the same data, while the output regulation equations are solved equivalently. Third, the robust formation controller and feasibility condition are further proposed based on previous learning results. Stability of the synthetical data-driven controller is analyzed under internal uncertainties and external disturbances. Finally, simulation results are provided to demonstrate the effectiveness of the hierarchical control framework. •Group formation tracking control under unknown environments is generally studied.•A hierarchical estimation-learning-control framework is proposed and analyzed.•Reinforcement learning is extended to learn both controllers and essential dynamics.•The data-driven property facilitates a unified design for networked control.