Virtual and augmented reality: Human sensory‐perceptual requirements and trends for immersive spatial computing experiences

Building on several decades of research and development, the recent progress in virtual reality (VR) and augmented reality (AR) devices with spatial computing technologies marks a significant leap in human–computer interaction, with applications ranging from entertainment and education to e‐commerce and healthcare. Advances in these technologies promise immersive experiences by simulating and augmenting the real world with computer‐generated digital content. The core objective of the VR and AR systems is to create convincing human sensory perceptions, thereby creating immersive and interactive experiences that bridge the gap between virtual and physical realities. However, achieving true immersion remains a goal, and it necessitates a comprehensive understanding of the neuroscience of human multisensory perception and accurate technical implementations to create a consistency between natural and synthetic sensory cues. This paper reviews the human sensory‐perceptual requirements vital for achieving such immersion, examines the current status and challenges, and discusses potential future advancements. A few examples of virtual and augmented reality systems over the past decades are shown. Top‐left: Sutherland's Sword of Damocles, 1968. Top‐right: NASA Ames Virtual Interface Environment Workstation, 1985. Middle: Oculus Rift, 2016. Bottom‐left: Intel Project Alloy, 2016, which was among the first all‐in‐one systems incorporating natural interactions. Bottom‐right: Apple Vision Pro, arguably the most advanced device as of 2024.