Multi-modal middle school experiment step detection method and system based on moving target semantic enhancement

The invention discloses a multi-modal middle school experiment step detection method and system based on motion target semantic enhancement, and the method comprises the steps: firstly carrying out the preprocessing of a video frame, obtaining a motion region through frame difference, obtaining a motion target through a target detection technology, and extracting a semantic time sequence feature through a BERT model; and performing time sequence dependence modeling on the video features in an encoder to obtain step-level visual time sequence features, fusing the step-level visual time sequence features with moving target semantic features in a decoder, constructing a relation between experiment steps and corresponding targets, and realizing accurate detection of the experiment steps in a middle school experiment video. The method can more effectively capture unique motion characteristics of the experimental steps, effectively distinguish different steps, realize accurate judgment of the experimental steps, an