Control of processing node operations

A data processing system and method wherein there is a plurality of processing nodes configured to perform operation to implement a multi-stage process over a plurality of time periods. A first of the plurality of processing nodes loads data required during a subsequent time period S1910, and a second of the plurality performs calculations using the loaded data S1920. At least one of the two processing nodes receives data from the other S1930 and during a subsequent time period performs calculations associated with a stage of the multi-stage process using the received data S1940. The processors may alternate between a compute phase and an exchange phase wherein data is exchanged between the processing nodes and each of the nodes executes a synchronisation instruction indicating a barrier between the compute and exchange phase wherein the data is received during the exchange phase. The data loaded from memory of the result of the computations may be weights of a neural network. Biases of a neural network or activations of nodes of a neural network.