Power Saving for Hardware Accelerated Applications with Dynamical Processor Switching

Services that require both heavy-load computation and low-latency have been increasing. To meet these requirements, an increasing number of servers are equipped with hardware accelerators such as a Graphics Processing Unit (GPU) or Field Programmable Gate Array (FPGA). These hardware accelerators can achieve higher performance computing and perform heavy-load computation at lower latency than CPUs. However, they often have fewer power-saving features than CPUs and tend to consume a constant amount of power regardless of computation-load variation. Thus, the problem of accelerators wasting power during low-load periods needs to be solved. In this paper, we propose a method in which when the computation load is so low that an accelerator is not needed, offloaded tasks are executed by the CPU without using an accelerator. On the basis of this idea, we design and implement middleware that enables an application to execute tasks by CPUs in low-load periods and offload them to an accelerator in high-load periods. We call this "processor switching". The proposed middleware can perform the processor switching transparently without modifying or suspending applications. We evaluate the proposed middleware in a usecase of vRAN applications, which have some of the strictest latency requirements among usecases involving accelerators. The results of the evaluations demonstrate that the proposed middleware can reduce power consumption of an accelerator in low-load periods by processor switching and the switching does not cause suspensions of an application. Furthermore, the latency overhead in offloading tasks does not violate the strict latency requirements.