Large-scale network public opinion-oriented Elasticsearch retrieval optimization system

The invention provides an Elasticsearch retrieval optimization system oriented to large-scale network public opinions. The Elasticsearch retrieval optimization system comprises a data aggregation module, an optimization mechanism and a retrieval service module, wherein the data aggregation module is...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: LIU JIALIN, LIU GUANGCHI, LIU ZHE, LI HUIKE, HE CHENGLONG, GU XUEHAI, MENG LINGWU, DING CAN, YIN XIAOYANG
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The invention provides an Elasticsearch retrieval optimization system oriented to large-scale network public opinions. The Elasticsearch retrieval optimization system comprises a data aggregation module, an optimization mechanism and a retrieval service module, wherein the data aggregation module is used for sending intermediate data obtained by preprocessing network public opinion multi-modal data to a distributed message bus Kafka, and finally persistently storing the intermediate data in an Elasticsearch distributed retrieval engine; the optimization mechanism comprises the following steps: constructing a text semantic vector based on a deep learning model SBert for realizing semantic retrieval; converting a text and a picture in the network public opinion multi-modal data into a text vector and a picture vector based on a CLIP multi-modal comparison learning model, wherein the text vector and the picture vector are used for vector retrieval; the retrieval performance of the Elasticsearch distributed retri