ACTION-OBJECT RECOGNITION IN CLUTTERED VIDEO SCENES USING TEXT

A mechanism is provided to implement an action-object interaction detection mechanism for recognizing actions in cluttered video scenes. An object bounding box is computed around an object of interest identified in a corresponding label in an initial frame where the object of interest appears in the frame. The object bounding box is propagated from the initial frame to a subsequent frame. For the initial frame and the subsequent frame: the object bounding boxes of the initial frame and the subsequent frame are refined and cropped based on the associated refined object bounding boxes. The set of cropped frames are processed to determine a probability that an action that is to be verified from the corresponding label is being performed. Responsive to determining the probability is equal to or exceeds a verification threshold, a confirmation is provided that the action-object interaction video performs the action that is to be verified.