Comparing Designs for Choice Experiments: A Case Study

This paper describes an empirical comparison of the performance of four designs for a discrete choice experiment. These designs were chosen to represent the range of construction techniques that are currently popular for choice experiments when no prior knowledge of the parameters is available. Each design had 320 respondents who each completed 16 choice sets. The results suggest that for the multinomial logit model (MNL) the design that is used at this stage is fairly unimportant. As the sample size gets smaller, however, differences between the designs become apparent. We also analysed the results using four different models which accommodate preference heterogeneity. We find that any of these models are able to predict choices more accurately for born in-sample and out-of-sample than the MNL model for the designs used here, and that the differences across designs arc larger for models with more parameters, although preliminary results suggest the gain appears to depend on the underlying preference structure.