Modeling Intrinsic Motivation in ACT-R: Mechanism of Intellectual Curiosity based on Pattern Matching

To date, many studies concerned with intrinsic motivation in humans and artificial agents based on a reinforcement learning framework have been conducted. However, these studies have rarely explained the correspondence between intrinsic motivation and other essential cognitive functions. This study aims to build a method to express curiosity in new environments via the ACT-R (Adaptive Control of Thought-Rational) cognitive architecture. To validate the effectiveness of this proposal, we implement several models of varying complexity using the method, and we confirm that the model’s behavior is consistent with human learning. This method focuses on the“ production compilation” and ”utility” modules, which are generic functions of ACT-R. It regards pattern matching with the environment as a source of intellectual curiosity. We prepared three cognitive models of path planning representing different levels of thinking. We made them learn in multiple-breadth maze environments while manipulating the strength of intellectual curiosity, which is a type of intrinsic motivation. The results showed that intellectual curiosity in learning an environment negatively affected the model with a shallow level of thinking but was influential on the model with a deliberative level of thinking. We consider the results to be consistent with the psychological theories of intrinsic motivation. Furthermore, we implemented the model using a conventional reinforcement learning agent and compared it with the proposed method.