Robot gaining accurate pouring skills through self-supervised learning and generalization

Pouring is one of the most commonly executed tasks in humans’ daily lives, whose accuracy is affected by multiple factors, including the type of material to be poured and the geometry of the source and receiving containers. In this work, we propose a self-supervised learning approach that learns the pouring dynamics, pouring motion, and outcomes from unsupervised demonstrations for accurate pouring. The learned pouring model is then generalized by self-supervised practicing to different conditions such as using unaccustomed pouring cups. We have evaluated the proposed approach first with one container from the training set and four new but similar containers. The proposed approach achieved better pouring accuracy than a regular human with a similar pouring speed for all five cups. Both the accuracy and pouring speed outperform state-of-the-art works. We have also evaluated the proposed self-supervised generalization approach using unaccustomed containers that are far different from the ones in the training set. The self-supervised generalization reduces the pouring error of the unaccustomed containers to the desired accuracy level. •Self-supervised learning from human demonstrations for accurate robotic pouring.•Peephole LSTM recurrent neural network for robot control which achieves human-like pouring accuracy and speed.•Generalization by self-supervised practicing which fine-tunes the model using the actual outcomes of the robot.•Extensive experiments on the generalization of the model to different containers, liquids, and solid materials.