Method and apparatus for cross-modal predictive coding for talking head sequences

A method and apparatus for transmitting and remotely displaying the audio and visual portion of a person speaking so that the audio and visual signals are synchronized. The audio signal is constantly transmitted to the receiver and is also used to create a predicted image of the lips of the talking...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Hauptverfasser:	CHEN, TSUHAN, RAO, RAM R
Format:	Patent
Sprache:	eng
Schlagworte:	ACOUSTICS CALCULATING COMPUTING COUNTING ELECTRIC COMMUNICATION TECHNIQUE ELECTRICITY IMAGE DATA PROCESSING OR GENERATION, IN GENERAL MUSICAL INSTRUMENTS PHYSICS PICTORIAL COMMUNICATION, e.g. TELEVISION SPEECH ANALYSIS OR SYNTHESIS SPEECH OR AUDIO CODING OR DECODING SPEECH OR VOICE PROCESSING SPEECH RECOGNITION
Online-Zugang:	Volltext bestellen
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Beschreibung
Zusammenfassung:	A method and apparatus for transmitting and remotely displaying the audio and visual portion of a person speaking so that the audio and visual signals are synchronized. The audio signal is constantly transmitted to the receiver and is also used to create a predicted image of the lips of the talking head. The actual lip image is compared to the predicted lip image. Based upon this comparison, it is determined which of three signals is to be transmitted to the receiver: no signal corresponding to the video signal, a signal corresponding only to the differences between the actual lip image and a predicted lip image, or the actual lip image. The receiver reconstructs a lip image based upon the audio signal received and the signal received, if any, corresponding to the video image and inserts it into the previously received video frame or modifies the previous frame accordingly.