Parallel processing system using dual port memories to communicate between each processor and the public memory bus

A parallel processing system 13 has a number of processing elements 16 connected to a public memory bus 18 via multiple port memories 10. One port of each memory is connected to the bus. The other ports are connected to processors. The multiple port memory is configured as a shared memory resource, which can be accessed both by the processors connected to the ports of the memory and by processors connected to the public memory bus. The memory may be divided into segments based on the data path width and the size of the memory. The system may comprise a number of separate nodes 11, 12, each of which has a number of processing elements and a number of multiple port memories.