Construction method of document vector search engine based on decoder-only architecture

The invention discloses a construction method of a document vector search engine based on decoder-only architecture, and relates to the technical field of document search engines, the method comprises the following steps: constructing enterprise document retrieval training data based on a large lang...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: ZHOU XINCHI, CONG WENLIN, YANG MING, ZHAO XIANGYANG, WEI GENGCHEN
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The invention discloses a construction method of a document vector search engine based on decoder-only architecture, and relates to the technical field of document search engines, the method comprises the following steps: constructing enterprise document retrieval training data based on a large language model; constructing an embedded vector generator with a decoder-only architecture large model; using enterprise document retrieval training data to train and optimize the embedded vector generator of the decoding-only architecture; calculating an embedded vector of the enterprise document by using the trained embedded vector generator of the decoding-only architecture, and storing the embedded vector in a vector database; and building a search engine background based on the vector database. According to the method, an advanced decoder-only architecture large model is used as a base, training data is constructed by utilizing a large language model, a search engine special for enterprise document contents is tra