All Roads Lead to Rome? Evaluating Value Elicitation Methods

Across five studies (N = 2,941), theoretically equivalent elicitation methods yield different estimates of consumers' willingness-to-pay. The Multiple Price List format results in lower WTP compared to the open-ended response format. Consumer demand estimated using the Becker-DeGroot-Marschak method is lower than demand elicited using real choice at a fixed price. How much people value a product or service is fundamentally important to researchers in economics, psychology, and marketing. Valuation can be assessed in many ways, including choice, hypothetical willingness-to-pay (HWTP), and incentivized willingness-to-pay measures, e.g., the Becker-DeGroot-Marschak (BDM) and Multiple Price List (MPL) methods. BDM and MPL are widely used to ensure incentive compatibility and encourage truthful responses. Valuations elicited using these two methods are considered to reflect consumers' "true" valuations and predict their behaviors in the real world. In this research, we challenge this assumption and find that these two elicitation methods systematically lower people's valuation. Specifically, we propose and find that valuation is highest when elicited through hypothetical (unincentivized) WTP, lower when elicited through the open-ended BDM method, and lowest when elicited through the MPL method. More importantly, demand estimated using both BDM and MPL is lower compared to demand estimated using real choice at a fixed price. This casts doubt on the presumed incentive compatibility of the BDM/MPL methods, which may produce misleadingly low WTP estimates. We report four preregistered and incentive-compatible studies (total N = 2,607) that test our predictions regarding the differences between value elicitation methods. In Study 1, 332 laboratory participants indicated the highest amount of money they were willing to pay to watch a movie. Participants were randomly assigned to one of three between-subjects conditions: Hypothetical Willingness-to-Pay (HWTP), the BDM, and the MPL methods. In the HWTP condition, participants answered, "What is the highest amount of money (in US dollars) you would be willing to pay to watch [movie-name]?" by entering a number into a textbox. In the BDM condition, participants first read instructions and answered a practice question to make sure they understood the BDM procedures. Then they answered the same open-ended WTP question as in the HWTP condition. Similarly, in the MPL condition, participants first read instructions on the MPL