Resource needs prediction in virtualized systems: Generic proactive and self-adaptive solution

Resource management of virtualized systems in cloud data centers is a critical and challenging task due to the fluctuating workloads and complex applications in such environments. Over-provisioning is a common practice to meet service level agreement requirements, but this leads to under-utilization of resources and energy waste. Thus, provisioning virtualized systems with resources according to their workload demands is essential. Existing solutions fail to provide a complete solution in this regard, as some of them lack proactivity and dynamism in estimating resources, while others are environment- or application-specific, which limits their accuracy in the case of bursty workloads. Effective resource management requires dynamic and accurate prediction. This work presents a novel prediction algorithm, which (1) is generic, and can thus be applied to any virtualized system, (2) is able to provide proactive estimation of resource requirements through machine learning techniques, and (3) is capable of real-time adaptation with padding and prediction adjustments based on prediction error probabilities in order to reduce under- and over-provisioning of resources. In several virtualized systems, and under different workload profiles, the experimental results show that our proposition is able to reduce under-estimation by an average of 86% over non-adjusted prediction, and to decrease over-estimation by an average of 67% versus threshold-based provisioning. •We propose a novel algorithm for generic, dynamic and multi-step ahead prediction of resource needs in virtualized systems.•We provide dynamic and adaptive prediction adjustment and a padding strategy to reduce the resource under/over-estimation.•We provide the optimal size of the sliding window and predicted data minimizing prediction errors through Genetic Algorithm.•The evaluation results show that on average, our algorithm reduces the under-estimation by 86% over non-adjusted prediction.•The results show that on average, our algorithm reduces the over-estimation by 67% over threshold-based provisioning.