Multi-task language model-oriented meta-knowledge fine tuning method and platform

A multi-task language model-oriented meta-knowledge fine tuning method and a platform. On the basis of cross-domain typical fractional learning, the method obtains highly transferable common knowledge on different data sets of similar tasks, i.e. meta-knowledge, mutually correlates and mutually stre...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Hongsheng Wang, Haijun Shan, Shengjian Hu
Format: Patent
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:A multi-task language model-oriented meta-knowledge fine tuning method and a platform. On the basis of cross-domain typical fractional learning, the method obtains highly transferable common knowledge on different data sets of similar tasks, i.e. meta-knowledge, mutually correlates and mutually strengthens learning processes of similar tasks in different domains corresponding to different data sets, thereby improving fine tuning effects of similar downstream tasks on data sets in different domains in language model applications, and improving parameter initialization abilities and generalization abilities of universal language models for similar tasks. The method is to fine tune a cross-domain data set of downstream tasks. Effects of a compression model obtained by means of fine tuning are not limited to a specific data set of such tasks. On the basis of a pre-training language model, the downstream tasks are fine tuned by means of using a meta-knowledge fine tuning network to obtain a language model of similar downstream tasks independent of data sets.