Dynamic power management for value-oriented schedulers in power-constrained HPC system

High performance computing (HPC) systems are confronting the challenge of improving their productivity under a system-wide power constraint in the exascale era. To measure the productivity of an HPC job, researchers have proposed to assign a monotonically decreasing time-dependent value function, called job-value, to that job. These job-value functions are used by the value-based scheduling algorithms to maximize the system productivity where system productivity is the accumulation of job-value for the completed jobs. In this study, we first show that the relative performance of the competing state-of-the-art static power allocation strategies interchange based on the level of the power constraint when applied to the value-based algorithms. We then investigate the limitations of these static strategies by relating the job completion rate to the resource utilization, and expose that there is non-negligible amount of unused resources for the scheduler to utilize. Even though the system is oversubscribed, these unused resources are insufficient to schedule new high-value jobs. Based on this observation, we propose a novel dynamic power management strategy for the value-based algorithms. Our dynamic allocation policy maximizes the system productivity, resource utilization, and job completion rate by utilizing application power-performance models to reallocate power from running jobs to newly arrived jobs. We simulate a large-scale system that uses job arrival traces from a real HPC system. We demonstrate that the dynamic-variant of each value-based algorithm earns up to 16% higher productivity and completes 13% more jobs compared to its static variants when power becomes a highly constrained resource in the system.