Spark-Streaming text similarity analysis-based data processing method and device
The invention provides a Spark-Streaming text similarity analysis-based data processing method and device, and relates to the technical field of computers. The method comprises the following steps of:dynamically obtaining a real-time text database; obtaining first text information and second text in...
Gespeichert in:
Hauptverfasser: | , , , |
---|---|
Format: | Patent |
Sprache: | chi ; eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | The invention provides a Spark-Streaming text similarity analysis-based data processing method and device, and relates to the technical field of computers. The method comprises the following steps of:dynamically obtaining a real-time text database; obtaining first text information and second text information according to the real-time text database; obtaining text length similarity information; obtaining text word sequence similarity information; obtaining text keyword similarity information; obtaining text grammar similarity information; and determining a statement similarity between the first text information and the second text information according to the text length similarity information, the text word sequence similarity information, the text keyword similarity information and thetext grammar similarity information. According to the method and device, the technical problem that rapid and correct emotion analysis cannot be carried out real-time network data in the prior art issolved, and the technical e |
---|