Multi-Camera Lighting Estimation for Mobile Augmented Reality

Lighting estimation is an important and long-standing task in computer vision and graphics communities[1]. Over the past few years, we have observed a rising demand for creating immersive user experiences in mobile augmented reality (AR) applications. In particular, environment lighting estimation is essential to render visually coherent virtual objects that need to be overlaid on physical scenes in mobile AR. In this work, we explore the key research questions in lighting understanding for an emerging application domain, mobile AR on multi-camera handheld devices, particularly try-on applications where end users leverage handheld mobile devices, such as smartphones, to overlay products of interest on their faces. Mobile AR try-on apps promote online shopping and often require photorealism to provide user experiences on par with physical try-on.