The Value of Knowledge: Joining Reward and Epistemic Certainty Optimisation for Anxiety-Sensitive Planning

Anxiety is one of the most basic emotional states and also the most common disorder. AI agents however are typically focused on maximising performance, concentrating on expected values and disregarding the degree of exposure to uncertainty. This paper introduces a formalism derived from Partially Observable Markov Decision Processes (POMDPs) to give the first model based on cognitive psychology of the anxiety induced by epistemic uncertainty (i.e. the lack of precision of knowledge about the current state of the world). An algorithm to generate policies balancing reward maximisation and anxiety reduction is given. It is then used on a classical example to demonstrate how this can lead in some cases to a dramatic reduction of epistemic uncertainty for nearly no cost and thus a more human-friendly reward optimisation. The empirical validation shows results reminiscent of behaviours that cognitive psychology identifies as coping mechanisms to anxiety.