Self-adaptive LDA topic model training system based on public opinion real-time data stream

The invention provides a self-adaptive LDA topic model training system based on public opinion real-time data flow. The self-adaptive LDA topic model training system comprises a data aggregation module, a data preprocessing module, a self-adaptive LDA model training module and an incremental LDA mod...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: LIU ZHE, LI HUIKE, HE CHENGLONG, GU XUEHAI, MENG LINGWU, LUO JUNZHOU, DING CAN, YIN XIAOYANG
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The invention provides a self-adaptive LDA topic model training system based on public opinion real-time data flow. The self-adaptive LDA topic model training system comprises a data aggregation module, a data preprocessing module, a self-adaptive LDA model training module and an incremental LDA model fusion module. The data aggregation module is used for extracting, converting and loading structured and semi-structured data, and inputting the structured and semi-structured data into a distributed message bus kafka; the data preprocessing module is used for preprocessing data in the message bus kafka and finally forming a weighted word vector; the adaptive LDA model training module is used for training to obtain LDA model results and merging the training results; and the incremental LDA model fusion module is used for carrying out fusion training to generate a new round of LDA model. The method is superior to a traditional LDA topic analysis method in accuracy and performance, is applied to actual engineering