TEXT AND AUDIO-BASED REAL-TIME FACE REENACTMENT

A computer-implemented method is disclosed. The method receives an input text (310) and a target image. The target image includes a target face. The method generates, based on the input text (310), a sequence of sets of acoustic features (625) corresponding to the input text (310). The method genera...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Hauptverfasser:	LUKIN, Maxim, MASHRABOV, Aleksandr, SAVCHENKOV, Pavel
Format:	Patent
Sprache:	eng ; fre ; ger
Schlagworte:	ACOUSTICS CALCULATING COMPUTING COUNTING MUSICAL INSTRUMENTS PHYSICS SPEECH ANALYSIS OR SYNTHESIS SPEECH OR AUDIO CODING OR DECODING SPEECH OR VOICE PROCESSING SPEECH RECOGNITION
Online-Zugang:	Volltext bestellen
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Beschreibung
Zusammenfassung:	A computer-implemented method is disclosed. The method receives an input text (310) and a target image. The target image includes a target face. The method generates, based on the input text (310), a sequence of sets of acoustic features (625) corresponding to the input text (310). The method generates, based on the sequence of sets of acoustic features (625), a sequence of sets of mouth key points (635). The method generates, based on the sequence of sets of mouth key points (635), a sequence of sets of facial key points (655). The method determines, based on the set sequence of sets of facial key points (655), a sequence of deformations of the target face. The method applies the sequence of deformations to the target image, thereby generating, a sequence of frames of an output video.