57‐1: Invited Paper: AirTouch System: Hand Skeleton Modelling for Augmented HMI Interactions

This paper presents a high‐performance capacitive touch system enhanced by a 3D vision‐based engine, the "AirTouch". The system is based on multiple 3D cameras that analyze the 50 cm near‐field volume above the screen surface. A real‐time 50 Hz 3D reconstruction engine with millimetric precision enables the sensing and tracking of several users and multiple hands or objects. The system identifies a preliminary set of hand poses: open palm, closed fist and pointing. For the pointing pose, the pointing target is estimated. It also determines the hand location, angle and movement of the fingers through advanced algorithms. The system aims to characterize the skeleton key elements including joints and segments of the hand, and reconstruct its mechanical properties (movement restrictions, angle correlations, etc.).