Heterogeneous Networks of Workstations

Parallel systems built from commodity CPUs and networking devices (Networks of Workstations or NoW) are easily and economically constructed. Rapid technological change means that as elements are added to or replaced in such a system, it will inevitably become heterogeneous. This paper examines issues in effectively using all the processing power of a heterogeneous NoW system. In particular, it examines the use of two different parallel programming models: MPI, a popular message passing system and Cilk, an extension to C with dataflow semantics. We consider both the performance of the two systems for a set of simple benchmarks representative of real problems and the ease of programming these systems. We show that, in many cases, Cilk both performs better than MPI and, additionally, that it requires less programming effort to write programs that exploit the full power of a heterogeneous system.