Spatio-Temporal Relationship Cognitive Learning for Multi-Robot Air Combat

Relationship cognition is crucial to learning-based Multi-Robot Systems (MRSs). As an advanced application of MRSs for fierce confrontation, the relationships among autonomous air combat robots inherently present complex time-varying characteristics, which makes relationship cognition even more difficult. However, previous studies have only focused on spatial cooperative relationships, thus ignoring the potential impact of the temporal dynamics of relationships on long-term cooperative behaviors. To tackle this drawback, we propose a novel Multi-Agent Deep Reinforcement Learning (MADRL) based autonomous air combat robots collaboration algorithm, called Spatio-Temporal Aerial robots Relationship Co-Optimization (STARCO). STARCO formulates the complex dynamic relationship cognition problem into a spatio-temporal deep Graph Neural Network (GNN) learning problem. On this basis, we overcome the limitations of previous methods, by accurately capturing the key spatio-temporal patterns from aggressive air combat, and enable global collaborative decision-making through joint strategy optimization. An empirical study shows that STARCO outperforms several state-of-the-art MARL baselines by 24.6% in learning performance. We also demonstrate that STARCO is capable of evolving various cooperative strategies comparable to human expert knowledge.