Selective Information Communication Considering the Influence of Each Other’s Messages in Multi-Agent Reinforcement Learning

In this paper, we propose a communication mechanism for multi-agent reinforcement learning that selectively sends the information needed to solve a task in a cooperative task. We address the question of “what” information should be sent to other agents from observations with information on multiple attributes, taking into account communication with other agents. The proposed method introduces a mechanism called Selective messenger, which combines the decomposition and organization of information by the deep generative model VAE and the selection of information by the Attention mechanism, as a mechanism for sending communication. Experiments show that the proposed method performs better than conventional communication methods in tasks where images with information on multiple attributes are observations and past communication needs to be referenced. In addition, the proposed method shows minimal performance variation compared to other methods when the message size for communication changes. This suggests that the our method sends important information more compactly. In the experiment to evaluate the effectiveness of the Selective messenger mechanism, we verified the efficacy of both the VAE and Attention components. The results show that the VAE component successfully performed disentanglement on the observations, and is capable of organizing and representing the information of each attribute contained within these observations. Furthermore, it show that the Attention component is able to appropriately select information while taking into account messages sent from other agents.