SYSTEMS AND METHODS FOR AN ENCODER-DECODER BASED FRAMEWORK FOR CODE GENERATION AND UNDERSTANDING

SYSTEMS AND METHODS FOR AN ENCODER-DECODER BASED FRAMEWORK FOR CODE GENERATION AND UNDERSTANDING

Embodiments described herein provide a mixture of encoder-decoder Transformer framework for multi-task pretraining and flexible finetuning for both code understanding and generation tasks. Specifically, the framework is built on multimodal encoder and decoder modules. During pre-training, the encode...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Wang, Yue, Le, Hung, Gotmare, Akhilesh Deepak, Li, Junnan, Hoi, Chu Hong
Format: Patent
Sprache:eng
Schlagworte:
CALCULATING
COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
COMPUTING
COUNTING
PHYSICS
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Embodiments described herein provide a mixture of encoder-decoder Transformer framework for multi-task pretraining and flexible finetuning for both code understanding and generation tasks. Specifically, the framework is built on multimodal encoder and decoder modules. During pre-training, the encoder-decoder framework is trained with multiple learning objectives, including a diverse set of self-supervised tasks over two major stages of pretraining on unimodal and bimodal data.