Hierarchical control of multi-agent reinforcement learning team in real-time strategy (RTS) games

Coordinated control of multi-agent teams is an important task in many real-time strategy (RTS) games. In most prior work, micromanagement is the commonly used strategy whereby individual agents operate independently and make their own combat decisions. On the other extreme, some employ a macromanagement strategy whereby all agents are controlled by a single decision model. In this paper, we propose a hierarchical command and control architecture, consisting of a single high-level and multiple low-level reinforcement learning agents operating in a dynamic environment. This hierarchical model enables the low-level unit agents to make individual decisions while taking commands from the high-level commander agent. Compared with prior approaches, the proposed model provides the benefits of both flexibility and coordinated control. The performance of such hierarchical control model is demonstrated through empirical experiments in a real-time strategy game known as StarCraft: Brood War (SCBW). •A hierarchical Command and Control model of RL agents for Real-Time Strategy Game.•The model enables the unit’s individual decisions while directed by a commander agent.•A self-organizing neural network realizes the agents at strategic and unit level.•Empirical works demonstrate the model’s flexibility in achieving agents’ coordination.