Self-adaptive LDA topic model training system based on public opinion real-time data stream
The invention provides a self-adaptive LDA topic model training system based on public opinion real-time data flow. The self-adaptive LDA topic model training system comprises a data aggregation module, a data preprocessing module, a self-adaptive LDA model training module and an incremental LDA mod...
Gespeichert in:
Hauptverfasser: | , , , , , , , |
---|---|
Format: | Patent |
Sprache: | chi ; eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | The invention provides a self-adaptive LDA topic model training system based on public opinion real-time data flow. The self-adaptive LDA topic model training system comprises a data aggregation module, a data preprocessing module, a self-adaptive LDA model training module and an incremental LDA model fusion module. The data aggregation module is used for extracting, converting and loading structured and semi-structured data, and inputting the structured and semi-structured data into a distributed message bus kafka; the data preprocessing module is used for preprocessing data in the message bus kafka and finally forming a weighted word vector; the adaptive LDA model training module is used for training to obtain LDA model results and merging the training results; and the incremental LDA model fusion module is used for carrying out fusion training to generate a new round of LDA model. The method is superior to a traditional LDA topic analysis method in accuracy and performance, is applied to actual engineering |
---|