Task processing method, device and equipment and storage medium

The invention discloses a task processing method, device and equipment and a storage medium, and belongs to the technical field of databases. The embodiment of the invention provides a method for realizing an optimization function by combining similar ETL tasks. Starting from the overall redundancy...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: QU TIANCI, WANG MENG
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The invention discloses a task processing method, device and equipment and a storage medium, and belongs to the technical field of databases. The embodiment of the invention provides a method for realizing an optimization function by combining similar ETL tasks. Starting from the overall redundancy condition of ETL tasks deployed on a platform, it is considered that SQL logic of each ETL task hasmutually repeated parts; the characteristics of each ETL task in the SQL logic aspect are extracted, classification is carried out according to the SQL characteristics of each ETL task, and thereforethe ETL tasks with the similar SQL logic are classified into the same class, and the function of automatically detecting the similar ETL tasks is achieved. ETL tasks with similar SQL (Structured QueryLanguage) logics are merged; therefore, the total number of the ETL tasks is reduced, and due to the fact that the number of the tasks to be executed is smaller, consumed time for executing the ETL tasks is saved, the overall