Knowledge-based person-centric human-robot interaction using facial and hand gestures

This paper presents a knowledge-based person-centric human-robot interaction system using facial and hand gestures. In the proposed method, face detection and person identification are first made. With the knowledge of the known user face and hand poses are then classified from the later image frame by the subspace method and the gestures are finally recognized. The rules for interpreting the gestures are selected according to each specific user recognized by the facial image. The user's name and gesture commands are sent to the robot through a Software Platform for Agent and Knowledge Management (SPAK) to implement person-centric human-robot interaction. The effectiveness of this method has been demonstrated by interacting with a humanoid robot Robovie.