Mind Games: Game Engines as an Architecture for Intuitive Physics

We explore the hypothesis that many intuitive physical inferences are based on a mental physics engine that is analogous in many ways to the machine physics engines used in building interactive video games. We describe the key features of game physics engines and their parallels in human mental representation, focusing especially on the intuitive physics of young infants where the hypothesis helps to unify many classic and otherwise puzzling phenomena, and may provide the basis for a computational account of how the physical knowledge of infants develops. This hypothesis also explains several ‘physics illusions’, and helps to inform the development of artificial intelligence (AI) systems with more human-like common sense. Behavioral studies over more than two decades have shown that young infants have a rich understanding of the physics of the world, with general expectations about the dynamics of objects and substances. However, this knowledge is also incomplete and inaccurate, and develops importantly over the first years of life. Recent computational models can capture aspects of physical scene understanding by performing probabilistic inferences over representations similar to those used in video game physics engines, which enable players to interact realistically with objects in virtual physical scenes. Game engines rely on numerous shortcuts and hacks to efficiently simulate approximations to Newtonian mechanics for complex scenes in real-time. Mental physics engines, solving a similar problem, may have converged on similar approximations and concepts. Physics-engine representations can help to explain the patterns of success and failure in the intuitive physics of infants, as well as illusions and misperceptions in adults. Probabilistic simulations in game physics engines are increasingly being used in the design of AI systems to enable common-sense reasoning about the physical world.