METHOD AND SYSTEM FOR MIXED REALITY INTERACTION WITH PERIPHERAL DEVICE

Methods and systems for providing a mixed reality (MR) interaction are provided. In one example, a method comprises: capturing, at a first time and using a camera of a head-mounted display (HMD) of a user, a first image of a physical interaction of the user with a physical object; measuring a movement of the HMD with respect to the physical object between the first time and a second time; processing the first image based on the measurement of the movement of the HMD to generate a second image; generating, based on the second image, a composite image of a virtual interaction involving the user; and displaying, via the HMD and based on the composite image, the virtual interaction in place of the physical interaction to the user at the second time.