Batch running method and device based on prepolymerization public table, and medium

The invention relates to a batch running method and device based on a prepolymerization public table and a medium. The method comprises the following steps that an SQL label task is obtained, an HIVE table on which the SQL label task depends is determined through analysis, division is conducted acco...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: TIE JINCHENG, LI MENGCONG
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The invention relates to a batch running method and device based on a prepolymerization public table and a medium. The method comprises the following steps that an SQL label task is obtained, an HIVE table on which the SQL label task depends is determined through analysis, division is conducted according to preset label logic, and a label batch running queue is obtained; aiming at the label batch queue, performing grouping according to the number of HIVE tables related to each label, obtaining a plurality of label groups, and obtaining a newly added label group prepolymerization SQL task according to the plurality of label groups; pre-aggregating SQL tasks according to the plurality of newly-added label groups, and obtaining a public table set; and performing batch running by using the public table set. Compared with the prior art, the method has the advantages that SQL task batch running time can be saved, unnecessary disk overhead increased by reading HIVE table data for multiple times in similar tasks is a