Batch running method and device based on prepolymerization public table, and medium
The invention relates to a batch running method and device based on a prepolymerization public table and a medium. The method comprises the following steps that an SQL label task is obtained, an HIVE table on which the SQL label task depends is determined through analysis, division is conducted acco...
Gespeichert in:
Hauptverfasser: | , |
---|---|
Format: | Patent |
Sprache: | chi ; eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | The invention relates to a batch running method and device based on a prepolymerization public table and a medium. The method comprises the following steps that an SQL label task is obtained, an HIVE table on which the SQL label task depends is determined through analysis, division is conducted according to preset label logic, and a label batch running queue is obtained; aiming at the label batch queue, performing grouping according to the number of HIVE tables related to each label, obtaining a plurality of label groups, and obtaining a newly added label group prepolymerization SQL task according to the plurality of label groups; pre-aggregating SQL tasks according to the plurality of newly-added label groups, and obtaining a public table set; and performing batch running by using the public table set. Compared with the prior art, the method has the advantages that SQL task batch running time can be saved, unnecessary disk overhead increased by reading HIVE table data for multiple times in similar tasks is a |
---|