Convolution engines for systolic neural network processor

Devices and methods for systolically processing data according to a neural network. A first processing unit performs computations of a first node of a first layer to generate a first output and attaches a first tag to the first output identifying the first processing unit. A second processing unit performs computations of a second node of the first layer to generate a second output and attaches a second tag to the second output identifying the second processing unit. A third processing unit performs computations of a third node of a second layer including receiving the first and second outputs, using a first convolutional engine to perform a first convolution on the first output using a first weight identified by the first tag, and using a second convolutional engine of to perform a second convolution on the second output using a second weight identified by the second tag.