Method and device for generating operation instruction sequence of robot

The invention provides a method for generating an operation instruction sequence of a robot. The method comprises the steps of obtaining an image feature sequence corresponding to a to-be-processed video; inputting the image feature sequence into a pre-trained hole convolutional neural network to obtain an action type sequence of the to-be-processed video; extracting a video clip corresponding toeach action type from the to-be-processed video, determining an operation object and an operated object of a candidate object in the video clip, and generating an action instruction corresponding to the action type of the video clip, an execution instruction corresponding to the operation object and a control instruction corresponding to the operated object; and forming an operation instruction combination by the action instruction, the operation instruction and the control instruction, and generating an operation instruction sequence of the to-be-processed video. The robot applying the methodprovided by the invention