Audio and video omnibearing simulation training method and system

The invention discloses an audio and video omnibearing simulation training method and system, and the method comprises the steps: collecting audio data and video data for executing simulation operation, obtaining the audio data, marking the audio features, carrying out the recognition of the ceremony behaviors of a seat person in the video data, obtaining a gesture and an action, and carrying out the recognition of the ceremony behaviors of the seat person. Non-linear image features of gestures and actions are obtained according to a cascade classifier; wherein the image features are obtained by outputting intermediate features of the gesture and the action according to the first stage of the cascade classifier, and taking the intermediate features as at least one input of the second stage of the cascade classifier; the audio features and the image features are fused, the emotional tendency of the seat personnel is detected according to the obtained fused features, the seat personnel are evaluated according t