Finite-Sample Variance of Local Polynomials: Analysis and Solutions

Fitting local polynomials in nonparametric regression has a number of advantages. The attractive theoretical features are in a partial contradiction to variance properties for random design and to practical experience over a broad range of situations. No upper bound can be given for the conditional variance. The unconditional variance is infinite when using optimal weights with compact support. Properties are better for Gaussian weights. We analyze local polynomials for finite sample size, both theoretically and numerically. It turns out that difficulties arise in sparse regions in the realization of the design, when the realization has locally a small variance and/or a skew empirical distribution. Two small-sample modifications of local polynomials are presented: local increase of bandwidth in sparse regions of the design, and local polynomial ridge regression. Both modifications combine a good finite-sample behavior with the asymptotic advantages of local polynomials.