Experimental Plans in Factorial Surveys: Random or Quota Design?

The factorial survey is an experimental design where respondents are asked to judge descriptions of varying situations (vignettes) presented to them. Combining the vignette variables (factors) and their levels is done by the researcher, who also takes the responsibility for getting an optimal design. To represent the universe of possible level combinations as accurately as possible, random designs are mostly used. A possible alternative are quoted designs. Up to now, there has been little discussion and only few research studies done about the pros and cons of random and quota samples for factorial surveys. The purpose of this article is to contribute to filling this gap. The conclusions drawn from the statistical considerations are illustrated by example analyses on the basis of fictitious data. Since the data structure produced by a factorial survey is a hierarchical one, the empirical analyses are carried out by using a multilevel program.