Distributed scheduling method and device, electronic equipment and computer storage medium

The invention provides a distributed scheduling method and device, electronic equipment and a computer storage medium. The distributed scheduling method is used for scheduling tasks in a big data platform, and the distributed scheduling method comprises the following steps: S1, determining the range...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: WAN SHILONG, FENG RUOYIN, ZOU XIAOFENG
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The invention provides a distributed scheduling method and device, electronic equipment and a computer storage medium. The distributed scheduling method is used for scheduling tasks in a big data platform, and the distributed scheduling method comprises the following steps: S1, determining the range of calculation tasks needing to be scheduled, and generating an initial task set; S2, performing layer-by-layer statistics on tasks of upstream reference objects of the tasks in all the initial task sets, and arranging the tasks according to an upstream and downstream sequence; S3, estimating the memory and processor overhead of each task, calculating respective scores through a resource overhead assessment algorithm, and sorting according to the scores to generate an execution task set; and S4, distributing the tasks in the execution task set to a plurality of computing nodes of the big data platform so as to enable the computing nodes to execute respective tasks. According to the distributed scheduling method pr