Quick cleaning and converting data extraction method
The invention discloses a quick cleaning and converting data extraction method, and relates to the technical field of data extraction. The method comprises: reading data content into a memory database, grouping the data content into a plurality of data segments, establishing multiple threads, alloca...
Gespeichert in:
Hauptverfasser: | , , |
---|---|
Format: | Patent |
Sprache: | chi ; eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | The invention discloses a quick cleaning and converting data extraction method, and relates to the technical field of data extraction. The method comprises: reading data content into a memory database, grouping the data content into a plurality of data segments, establishing multiple threads, allocating a to-be-processed data block to each thread, each thread independently cleaning and convertingthe data blocks, and combining and storing the converted data blocks into the memory database. The method comprises the following steps: judging whether a current data segment in a current thread is in summary association with other data blocks or not through a task scheduling processor; if the data blocks exist, summarizing the summarized and associated data blocks after the data blocks are cleaned, splitting the summarized and associated data blocks again to form new data blocks, and converting the new data blocks by adopting multiple threads; if not, directly cleaning and converting the current data block; finally, |
---|