Iterative, Multiple-Method Questionnaire Evaluation Research: A Case Study

This article summarizes a series of three biennial evaluations of a labor force questionnaire that collects data on worker displacement. Adopting a dichotomy for evaluation research that draws a distinction between questionnaire pretesting (developmental/pre-implementation evaluations) and quality assessment (post-implementation evaluations), the first two evaluations in the series represent quality assessment research. The third evaluation is somewhat unusual in that it can be classified as both pretesting and quality assessment research. Though the scope of work for each evaluation differed, three standard methods for evaluating questionnaires were used during each of these efforts: interviewer debriefings, interaction/behavior coding, and respondent debriefing. It should be noted that this series of studies was not iterative by design - it evolved as such due to unforeseen circumstances and, in the process, yielded unanticipated benefits.