A Multi-Tier Stacked Ensemble Algorithm to Reduce the Regret of Incremental Learning for Streaming Data

Incremental Learning (IL) is an exciting paradigm that deals with classification problems based on a streaming or sequential data. IL aims to achieve the same level of prediction accuracy on streaming data as that of a batch learning model that has the opportunity to see the entire data at once. The performance of the traditional algorithms that can learn the streaming data is nowhere comparable to that of batch learning algorithms. Reducing the regret of IL is a challenging task in real-world applications. Hence developing an innovative algorithm to improve the ILs performance is a necessity. In this paper, we propose a multi-tier stacked ensemble (MTSE) algorithm that uses incremental learners as the base classifiers. This novel algorithm uses the incremental learners to predict the results that get combined by the combination schemes in the next tier. The meta-learning in the next tier generalizes the output from the combination schemes to give the final prediction. We tested the MTSE with three data sets from the UCI machine learning repository. The results reveal that MTSE is superior in performance over the SE learning.