A Comparison of Machine Learning Algorithms for Automatic Cloud Resource Scaling on a Multi-Tenant Platform

In an online multi-tenant machine learning platform, the system manager would dynamically load the computing resource according to the tenant’s demand. With cloud computing services, the platform can rent or release computing resources dynamically to fulfill tenant usage which minimizes resource consumption and ensure scalability. Currently, many cloud-based services providers are using the rule-based, threshold auto-scaling mechanism. However, the rule-based method is not efficient, as the nature of availability and cost-reducing violate each other in this method, especially with the sudden increase or variation of the demand. In this paper, we compare several machine-learning-based predictive algorithms to build models based on the information of the system used to predict future usage demands. Decisions made based on this prediction save over 80% cloud resource consumption compared to the rule-based method.