TRAINING OF TEXT AND IMAGE MODELS

TRAINING OF TEXT AND IMAGE MODELS

A method of training a text model using a plurality of text passage combinations, each text passage combination comprising a respective first text passage and a respective second text passage describing a same matter as the respective first text passage but being differently worded than the respecti...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: NAUMANN, Tristan, SCHWAIGHOFER, Anton, POON, Hoifung, BANNUR, Shruthi Jaisimha, HYLAND, Stephanie, OKTAY, Ozan, COELHO DE CASTRO, Daniel, VALLE, Javier Alvarez, USUYAMA, Naoto, NORI, Aditya
Format: Patent
Sprache:eng ; fre ; ger
Schlagworte:
CALCULATING
COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
COMPUTING
COUNTING
ELECTRIC DIGITAL DATA PROCESSING
HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATIONTECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING ORPROCESSING OF MEDICAL OR HEALTHCARE DATA
INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTEDFOR SPECIFIC APPLICATION FIELDS
PHYSICS
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:A method of training a text model using a plurality of text passage combinations, each text passage combination comprising a respective first text passage and a respective second text passage describing a same matter as the respective first text passage but being differently worded than the respective first text passage. The training comprises minimizing a measure of statistical difference between a respective value of a first text embedding and the corresponding value of a second text embedding over the plurality of text passage combinations. The method then comprises jointly training the text model and an image model based on plurality of image-text combinations, each comprising a respective image and a respective textual report describing the respective image. The joint training comprises minimizing a measure of statistical difference between the value of an image embedding and the corresponding value of a third text embedding over the plurality of image-text combinations.