Elemental Subsets: The Building Blocks of Regression

In a regression dataset an elemental subset consists of the minimum number of cases required to estimate the unknown parameters of a regression model. The resulting elemental regression provides an exact fit to the cases in the elemental subset. Early methods of regression estimation were based on combining the results of elemental regressions. This approach was abandoned because of its computational infeasibility in all but the smallest datasets and because of the arrival of the least squares method. With the computing power available today, there has been renewed interest in making use of the elemental regressions for model fitting and diagnostic purposes. In this paper we consider the elemental subsets and their associated elemental regressions as useful "building blocks" for the estimation of regression models. Many existing estimators can be expressed in terms of the elemental regressions. We introduce a new classification of regression estimators that generalizes a characterization of ordinary least squares (OLS) based on elemental regressions. Estimators in this class are weighted averages of the elemental regressions, where the weights are determined by leverage and residual information associated with the elemental subsets. The new classification incorporates many existing estimators and provides a framework for developing new alternatives to least squares regression, including the trimmed elemental estimators (TEE) proposed in this paper.