PERIODICALLY COOPERATIVE MULTI-AGENT REINFORCEMENT LEARNING

Disclosed herein are methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for modeling agents in multi-agent systems as reinforcement learning (RL) agents and training control policies that cause the agents to cooperate towards a common goal. A method can include generating, for each of a group of simulated local agents in an agent network in which the simulated local agents share resources, information, or both, experience tuples having a state for the simulated local agent, an action taken by the simulated local agent, and a local result for the action taken, updating each local policy of each simulated local agent according to the respective local result, providing, to each of the simulated local agents, information representing a global state of the agent network, and updating each local policy of each simulated local agent according to the global state of the agent network.