Progress in Programming the HRP-2 Humanoid Using Spoken Language

The current research analyses and demonstrates how spoken language can be used by human users to communicate with the HRP-2 humanoid to program the robot's behavior in a cooperative task. The task involves the humans and the HRP-2 working together to assemble a piece of furniture. The objectives of the system are to 1) Allow the human to impart knowledge of how to accomplish a cooperative task to the robot, i.e. to program the robot, in the form of a sensory-motor action plan. 2) To do this in a semi-natural and real-time manner using spoken language. In this framework, a system for spoken language programming (SLP) is presented, and experimental results are presented from this prototype system. In Experiment 1, the human programs the robot to assist in assembling a small table. In Experiment 2, the generalization of the system is demonstrated as the user programs the robot to assist in taking the table apart. The SLP is evaluated in terms of the changes in efficiency as revealed by task completion time and number of command operations required to accomplish the tasks with and without SLP. Lessons learned are discussed, along with plans for improving the system, including developing a richer base of robot action and perception predicates that will allow the use of richer language. We thus demonstrate - for the first time - the capability for a human user to tell a humanoid what to do in a cooperative task so that in real time, the robot performs the task, and acquires new skills that significantly facilitate the cooperative human-robot interaction.