On Bayesian integration in sensorimotor learning: Another look at Kording and Wolpert (2004)

Kording and Wolpert (2004), hereafter referred to as KW, describe an experiment where subjects strove for accuracy in a stochastic environment and, on some trials, received mid-trial and post-trial feedback. KW claims that subjects learned the underlying stochastic distribution from the post-trial feedback of previous trials. KW also claims that subjects regarded mid-trial feedback that had a smaller visual size as more precise and they were therefore more sensitive to such mid-trial feedback. KW concludes that the observations are consistent with optimal Bayesian learning. KW has become an extremely influential paper in the large literature arguing that subjects are optimal Bayesian learners in stochastic environments. It is therefore crucial that the KW conclusions follow from their dataset. We note that KW analyzes data that have been both averaged across trials and averaged across other important trial-specific details. We also note that KW mischaracterizes the accuracy of the mid-trial feedback and the relative sizes of the mid-trial feedback. When we analyze the trial-level KW data, we do not find that subjects were more sensitive to mid-trial feedback when it had a smaller visual size. Our trial-level analysis also suggests a recency bias, rather than evidence that the subjects learned the stochastic distribution. In other words, we do not find that the observations are consistent with optimal Bayesian learning. In the KW dataset, it seems that evidence for optimal Bayesian learning is a statistical artifact of analyzing averaged data. Our results from the KW dataset would seem to have important implications for the literature on Bayesian judgments.