Towards seamless human robot collaboration: integrating multimodal interaction

This paper discusses the challenges in the collaboration between human operators and industrial robots for assembly operations focusing on safety and simplified interaction. A case study is presented, involving perception technologies for the robot in conjunction with wearable devices used by the operator. In terms of robot perception, a manual guidance module, an air pressor contact sensor namely skin, and a vision system for recognition and tracking of objects have been developed and integrated. Concerning the wearable devices, an advanced user interface including audio and haptic commands accompanied by augmented reality technology are used to support the operator and provide awareness by visualizing information related to production and safety aspects. In parallel, safety functionalities are implemented through collision detection technologies such as a safety skin and safety monitored regions delimiting the area of the robot activities. The complete system is coordinated under a common integration platform and it is validated in a case study of the white goods industry.