Benchmarking parallel programs in a multiprogramming environment: The PAR-bench system

While the efficiency of multitasking is proven for parallel programs running in a dedicated environment, this paper wants to show a new approach for the assessment of multitasking. It describes the benchmark generation environment PAR-Bench, which enables measurements of effects introduced by parallel programs running in a multiprogramming mode. The PAR-Bench system is implemented on Cray multiprocessor systems under the operating systems COS and UNICOS. Using PAR-Bench, the benchmark process is divided into two parts: In a first step, according to user-supplied parameters like MFLOPS rate, memory and I/O activities, CPU time etc., the PAR-Bench system generates synthetic benchmark programs by using the hardware performance monitor HPM. These programs can be used to simulate a given site's workload in a flexible way. In a second step, the system can be used to run this workload several times with varied parameters; the substantial work to be done is fixed, nevertheless dynamical changes of program parameters like memory size, priority, and also variations of the degree of parallelism are supported. The PAR-Bench system provides information about mutual influences of parallel programs and background load, enabling us to evaluate different multitasking implementations, different operating systems, and different computer hardware. Because of the abundance of data concerning characteristic program parameters and program timings, as a further component of PAR-Bench the graphical analyzing system GRANSYS was developed, realizing automatic analysis features for benchmark data including data interpretation and visualization.