An energy, performance efficient resource consolidation scheme for heterogeneous cloud datacenters

Datacenters are the principal electricity consumers for cloud computing that provide an IT backbone for today's business and economy. Numerous studies suggest that most of the servers, in the US datacenters, are idle or less-utilised, making it possible to save energy by using resource consolidation techniques. However, consolidation involves migrations of virtual machines, containers and/or applications, depending on the underlying virtualisation method; that can be expensive in terms of energy consumption and performance loss. In this paper, we: (a) propose a consolidation algorithm which favours the most effective migration among VMs, containers and applications; and (b) investigate how migration decisions should be made to save energy without any negative impact on the service performance. We demonstrate through a number of experiments, using the real workload traces for 800 hosts, approximately 1516 VMs, and more than million containers, how different approaches to migration, will impact on datacenter's energy consumption and performance. We suggest, using reasonable assumptions for datacenter set-up, that there is a trade-off involved between migrating containers and virtual machines. It is more performance efficient to migrate virtual machines; however, migrating containers could be more energy efficient than virtual machines. Moreover, migrating containerised applications, that run inside virtual machines, could lead to energy and performance efficient consolidation technique in large-scale datacenters. Our evaluation suggests that migrating applications could be ~5.5% more energy efficient and ~11.9% more performance efficient than VMs migration. Further, energy and performance efficient consolidation is ~14.6% energy and ~7.9% performance efficient than application migration. Finally, we generalise our results using several repeatable experiments over various workloads, resources and datacenter set-ups. •A consolidation method is suggested to manage datacenter resources – particularly, when container run inside VMs.•We consider various approaches to migrate VMs, containers and VMs plus containers (applications).•We model heterogeneity of datacenter resources (hosts), performance of VM, container migrations, as well as applications.•The “containerCloudSim” simulator is extended to make containers run inside VMs, perform VMs and containers migrations.•Using real workload traces, we suggest application migration more energy, performance efficient th