Compressed data direct calculation method and system based on nonvolatile storage
The invention relates to a compressed data direct calculation method and system based on nonvolatile storage. The method comprises the following steps that compressed text data and analysis application based on the compressed text data are obtained; performing redundancy elimination on the compresse...
Gespeichert in:
Hauptverfasser: | , , |
---|---|
Format: | Patent |
Sprache: | chi ; eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | The invention relates to a compressed data direct calculation method and system based on nonvolatile storage. The method comprises the following steps that compressed text data and analysis application based on the compressed text data are obtained; performing redundancy elimination on the compressed text data, allocating space on a memory pool, and constructing a DAG data structure of nonvolatile storage; and traversing the DAG data structure stored in a nonvolatile manner by adopting a preset traversing strategy to obtain a text analysis result. According to the method, the data locality can be improved, and the write operation on the nonvolatile storage is reduced, so that the efficiency is improved. Therefore, the method can be widely applied to the technical field of big data processing.
本发明涉及一种基于非易失性存储的压缩数据直接计算方法和系统,包括以下步骤:获取压缩文本数据和基于该压缩文本数据的分析应用;对压缩文本数据进行冗余消除,并在内存池上分配空间,构建非易失性存储的DAG数据结构;采用预设遍历策略对非易失性存储的DAG数据结构进行遍历,得到文本分析结果。本发明可以提高数据局部性,降低对非易失性存储的写操作,从而提高效率。因此,本发明可以广泛应用于大数据处理技术领域。 |
---|