VIRTUAL INTERACTION WITH IMAGE PROJECTION

Embodiments that relate to providing a low-latency interaction in a virtual environment are provided. In one embodiment an initial image of a hand and initial depth information representing an initial actual position are received. The initial image is projected into the virtual environment to an initial virtual position. A segmented version of the initial image is provided for display in the virtual environment at the initial virtual position. A subsequent image of the hand and depth information representing a subsequent actual position are received. The subsequent image is projected into the virtual environment to a subsequent virtual position. A segmented version of the subsequent image is provided for display at the subsequent virtual position. A collision is detected between a three-dimensional representation of the hand and a virtual or physical object.