Precision tracking of user interaction with a virtual input device

An augmented reality or virtual reality (AR/VR) system can include a virtual input device that can be rendered by an HMD, and a wearable impact detection device, such as a ring, smart watch, wristband, etc. with an inertial measurement unit (IMU), that can be used in conjunction with the HMD to track a location of the user's hands relative to the perceived location of the rendered virtual keyboard using, e.g., vision-based tracking via the HMD and determine when an intended input (e.g., button press) is entered by the user by detecting an impact of the user's finger(s) on a physical surface. The AR/VR system can then determine which key is pressed based on the physical location of the user's hands (e.g., using the vision-based tracking) and, more precisely, the user's finger(s) causing the detected impact and the closest key of the virtual input device to the detected point of impact.