Audio and video synchronization discrimination method and device, equipment and storage medium

The invention relates to the technical field of artificial intelligence, and discloses an audio and video synchronization discrimination method and device, equipment and a storage medium, and the method comprises the steps: generating a multi-modal feature vector based on an LSTM network, a video fe...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: ZHOU CHAOYONG, CHEN YUANXU, ZHOU CHEN, WU SHIBIN
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The invention relates to the technical field of artificial intelligence, and discloses an audio and video synchronization discrimination method and device, equipment and a storage medium, and the method comprises the steps: generating a multi-modal feature vector based on an LSTM network, a video feature vector, a text feature vector and a lip key point feature vector; generating a multi-dimensional audio feature vector based on the GRU network and the audio feature vector; and calculating probability distribution similarity, and when the probability distribution similarity meets a preset condition, judging that the target audio and the target video are synchronous. Through the above mode, the probability distribution similarity of the multi-modal feature vector fusing the lip key point feature vector and the text vector and the multi-dimensional audio feature vector is calculated, and the audio and the video are judged to be synchronous under the condition that the probability distribution similarity meets t