Apparatus and method for training model for document summarization

An apparatus for training a document summarization model includes a token generation unit, a named entity recognition unit, and a model training unit. The token generation unit generates document tokens and summarization tokens. The named entity recognition unit assigns named entity token status to...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Choi, Hyun-Jin, Lee, Hyun-Jae, Kim, Ju-Dong, Yun, Jae-Woong, Hwang, Bong-Kyu
Format: Patent
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:An apparatus for training a document summarization model includes a token generation unit, a named entity recognition unit, and a model training unit. The token generation unit generates document tokens and summarization tokens. The named entity recognition unit assigns named entity token status to a summarization token, recognized as a named entity through NER, and assigns non-named entity token status to the other tokens. The model training unit obtains feature vectors by inputting the plurality of document tokens into an encoder inside a document summarization model, obtains a first loss related to the named entity token, a second loss related to the other tokens, and a total loss using a weighted value by inputting the feature vectors, the summarization tokens, the named entity token, and the non-named entity token into a decoder inside the document summarization model, and trains the document summarization model on the basis of the total loss.