A Comparison of Subset Selection and Adaptive Basis Function Construction for Polynomial Regression Model Building

The approach of subset selection in polynomial regression model building assumes that the chosen fixed full set of predefined basis functions contains a subset that is sufficient to describe the target relation sufficiently well. However, in most cases the necessary set of basis functions is not known and needs to be guessed - a potentially non-trivial (and long) trial and error process. In our previous research we considered an approach for polynomial regression model building which is different from the subset selection - letting the regression model building method itself construct the basis functions necessary for creating a model of arbitrary complexity without restricting oneself to the basis functions of a predefined full model. The approach is titled Adaptive Basis Function Construction (ABFC). In the present paper we compare the two approaches for polynomial regression model building - subset selection and ABFC - both theoretically and empirically in terms of their underlying principles, computational complexity, and predictive performance. Additionally in empirical evaluations the ABFC is compared also to two other well-known regression modelling methods - Locally Weighted Polynomials and Multivariate Adaptive Regression Splines.