Control of data transfer between processing nodes

A data processing system 900 comprises nodes, each comprising a memory which stores an array 910 of data items, wherein each node belongs to at least two different sets 920, 930 of nodes. A processor of each node is configured to execute compute instructions during a compute phase and following a precompiled synchronisation barrier, enter an exchange phase, when it takes part in a reduce-scatter collective using the respective array of data items to obtain a reduced subset of an array of data items, wherein the reduce-scatter collective is performed between nodes of a different one of the respective different sets of processing nodes. Subsequently, the processor exchanges the respective reduce subset of the array of data items using an all-reduce collective with processing nodes of a further of the different sets of processing nodes to which the respective processing node belongs to obtain a further reduced subset of the array of data items. Subsequently, the processor takes part in an all-gather collective using the further reduced subset to obtain a reduced array of data items, wherein the all-gather collective is performed between nodes of one of the different ones of the respective sets of nodes.