Sim-to-real learning of 2D multiple sound source localization

A computer-implemented method is provided for training a multi-source sound localization model using labeled simulation data and unlabeled real data. The method includes inputting the labeled simulation data and the unlabeled real data respectively into a multi-source sound localization model of a n...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Hauptverfasser:	Le Moing, Guillaume Jean Victor Marie, Agravante, Don Joven Ravoy, Vinayavekhin, Phongtharin, Inoue, Tadanobu, Vongkulbhisal, Jayakorn, Munawar, Asim
Format:	Patent
Sprache:	eng
Schlagworte:	ACOUSTICS CALCULATING COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS COMPUTING COUNTING DEAF-AID SETS ELECTRIC COMMUNICATION TECHNIQUE ELECTRICITY LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKEACOUSTIC ELECTROMECHANICAL TRANSDUCERS MUSICAL INSTRUMENTS PHYSICS PUBLIC ADDRESS SYSTEMS SPEECH ANALYSIS OR SYNTHESIS SPEECH OR AUDIO CODING OR DECODING SPEECH OR VOICE PROCESSING SPEECH RECOGNITION
Online-Zugang:	Volltext bestellen
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Beschreibung
Zusammenfassung:	A computer-implemented method is provided for training a multi-source sound localization model using labeled simulation data and unlabeled real data. The method includes inputting the labeled simulation data and the unlabeled real data respectively into a multi-source sound localization model of a neural network to obtain a localization heatmap from an output layer of the multi-source sound localization model for each of the labeled simulation data and the unlabeled real data. The method further includes inputting the localization heatmap for each of the labeled simulation data and the unlabeled real data into an output discriminator. The method also includes training the output discriminator so that the output discriminator assigns a domain class label to distinguish simulation data from real data. The method additionally includes training, by a hardware process, the multi-source sound localization model by a first adversarial loss for the output discriminator with an original localization model loss.