Evaluation and Selection of Models for Attrition Nonresponse Adjustment

This article considers a longitudinal survey like the U.S. Survey of Income and Program Participation (SIPP), with successive "waves" of data collection from sampled individuals, in which nonresponse attrition occurs and is treated by weighting adjustment, either through adjustment cells or a model like logistic regression in terms of auxiliary covariates. We measure the biases in estimated initial-wave (Wave 1) attribute totals between the survey-weighted estimator in the first wave and the weight-adjusted estimator for the same Wave 1 item total based on later-wave respondents. Three new metrics of quality are defined for models used to adjust a longitudinal survey for attrition. The metrics combine estimated between-wave adjustment biases based on subsets of the sample, relative to the estimated total, for various survey items. The maximum of the biases for estimated totals of a survey item is calculated from the weight-adjusted subtotal of the first j sample units, as j ranges from 1 to the size of the entire (Wave 1) sample, after a random re-ordering either of the whole sample or of the units within distinguished cells (which are then also randomly reordered); and the average over re-orderings of the maximal adjustment bias is divided by the estimated Wave 1 attribute total to give the metric value. Confidence bands for the metrics are estimated, and the metrics are applied to judge the quality of and to select among a collection of logistic-regression models for attrition nonresponse adjustment in SIPP 96.