Meaning and reference from a probabilistic point of view

The rise of Bayesian models of cognition requires that traditional questions in epistemology and metaphysics, such as how models relate to reality and how one observer's models relate to another's, be reframed in probabilistic terms. In this paper we take up these questions beginning from a subjective (Bayesian) conception of probability, in which distinct observers hold potentially different probabilistic models of the world, with no one observer necessarily possessing the “true” one. The key question is what terms in a probabilistic theory mean—that is, what they refer to and what their truth conditions are. We address this question with tools from information theory. We introduce the translation uncertainty, a generalization of the Kullback–Leibler divergence that expresses the discrepancy between two observers’ probabilistic models of a common environment. We derive a number of basic information-theoretic relationships among observers, showing for example that the probability that two Bayesian observers will classify the world similarly (called the concordance) depends on the translation uncertainty between their respective models of the world. Our framework suggests a pathway to a semantics for a “probabilistic language of thought.”