Systolic neural network engine with crossover connection optimization

Devices and methods for systolically processing data according to a neural network. In one aspect, a first arrangement of processing units includes at least first, second, third, and fourth processing units. The first and second processing units are connected to systolically pulse data to one another, and the third and fourth processing units are connected to systolically pulse data to one another. A second arrangement of processing units includes at least fifth, sixth, seventh, and eighth processing units. The fifth and sixth processing units are connected to systolically pulse data to one another, and the seventh and eighth processing units are connected to systolically pulse data to one another. The second processing unit is configured to systolically pulse data to the seventh processing unit along a first interconnect and the third processing unit is configured to systolically pulse data to the sixth processing unit along a second interconnect.