Quick cleaning and converting data extraction method

The invention discloses a quick cleaning and converting data extraction method, and relates to the technical field of data extraction. The method comprises: reading data content into a memory database, grouping the data content into a plurality of data segments, establishing multiple threads, alloca...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: HE JING, HUO RONG, TANG TONGWEI
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The invention discloses a quick cleaning and converting data extraction method, and relates to the technical field of data extraction. The method comprises: reading data content into a memory database, grouping the data content into a plurality of data segments, establishing multiple threads, allocating a to-be-processed data block to each thread, each thread independently cleaning and convertingthe data blocks, and combining and storing the converted data blocks into the memory database. The method comprises the following steps: judging whether a current data segment in a current thread is in summary association with other data blocks or not through a task scheduling processor; if the data blocks exist, summarizing the summarized and associated data blocks after the data blocks are cleaned, splitting the summarized and associated data blocks again to form new data blocks, and converting the new data blocks by adopting multiple threads; if not, directly cleaning and converting the current data block; finally,