Data high-speed import method and system based on MPP database

The invention relates to a data high-speed import method and system based on an MPP database. The method comprises the steps that a data description file is acquired; a data file is read, and a corresponding md5 code is generated for the data file; whether the md5 code is consistent with an md5 code...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: ZHAN XIAN, YU LUNQIANG, HUANG RUITING, ZHENG YUHAO
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The invention relates to a data high-speed import method and system based on an MPP database. The method comprises the steps that a data description file is acquired; a data file is read, and a corresponding md5 code is generated for the data file; whether the md5 code is consistent with an md5 code in the data description file is judged; if not, a notice of collection non-passage is output; and if yes, data in the data file is allocated to corresponding data nodes according to the data description file. According to the data high-speed import method and system, the data description file is defined first to describe a collection protocol of the data file, the md5 code of the data file is collected and compared with the md5 code in the data description file, the data in the data file is collected and Hash values are calculated only under the condition that the md5 codes are consistent, and the Hash values of the data are allocated to the corresponding nodes for storage. Therefore, dataimport speed is increased