Fast distributed real-time data processing method

The invention discloses a fast distributed real-time data processing method. The fast distributed real-time data processing method uses a ring-like data structure to store data, the storage file size can be regulated through variables, and the new and old data coverage can be performed automatically...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: WANG JING, QIE DIANFU, CHEN JINMING, LIU CHANG, WANG YIRONG, XIE JIHUI, ZHANG JINGCHUAN, ZHU XI
Format: Patent
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The invention discloses a fast distributed real-time data processing method. The fast distributed real-time data processing method uses a ring-like data structure to store data, the storage file size can be regulated through variables, and the new and old data coverage can be performed automatically. The fast distributed real-time data processing method supports the partial receiving of data gathered by different data sources or data gathered by the same data source; through a memory index technology, the data query speed is more than 7 times faster than that of the traditional data query; a lot of data is accessed reliably and fast through multi-threading and data lock technologies, and special data compression algorithm and Zip compression algorithm are used to reduce data storage and improve data storage speed.