All-to-all message exchange in parallel computing systems

Disclosed is a method of all-to-all message exchange between program tasks in a parallel processing system. The program tasks are connected by an interconnection network with a hierarchical network topology comprising N>1 hierarchy levels in, n = 1 to N, in which a first level l1 comprises a plurality of groups of the tasks and. each higher level l(n>1) comprises at least one group of level l(n-1) groups. A unique hierarchical identifier is defined for each task, the hierarchical identifier being dependent on the groups to which that task belongs in respective hierarchy levels of the network topology. In each of successive phases of the exchange, each task sends a message via the network to a respective destination task whose hierarchical identifier is determined, in dependence on the hierarchical identifier of the sending task and the network topology, such that the resulting exchange pattern for the all-to-all message exchange exploits hierarchical distance in the network topology in a desired manner.