Generation of co-speech gestures of robot based on morphemic analysis

We propose a methodology for a robot to automatically generate felicitous co-speech gestures corresponding to robot utterances. First, the proposed method determines the part of a given robot utterance, where the robot makes a gesture by doing a morphemic analysis on the sentence of utterance. The part is herein called an expression unit. The method then predicts a gesture type to characterize the expression unit in the sense of conveying thoughts and feelings. The gesture type is selected from the four types of iconic, metaphoric, beat, and deictic categorized by McNeill by performing morphemic analysis on the sentence. A gesture proper to the gesture type is retrieved from a database of motion primitives that are built with predefined a limited number of words. For retrieving, Word2Vec is applied to estimate word similarity between the predefined words in the database and words in the expression unit such that the method can deal with an arbitrary sentence and generate an appropriate gesture for similar words in meaning. The proposed method showed 83% accuracy in determining expression units and gesture types for a set of sentences in Korean. Furthermore, a user study on feasibility has been performed with a humanoid, NAO, and received positive evaluations in terms of anthropomorphism for the robot. [Display omitted] •The proposed method determines parts of a sentence that can be executed by gestures with utterance. This part is herein called an expression unit.•Gesture types are then predicted to characterize the expression unit in the sense of conveying thoughts and feelings.•Appropriate gestures are determined for each expression unit considering gesture types.•Experiments are analyzed for accuracy comparison on determining expression units and gesture types. In addition, comparisons of effectiveness are also verified in terms of human–robot interaction.