Distributed scheduling method and device, electronic equipment and computer storage medium
The invention provides a distributed scheduling method and device, electronic equipment and a computer storage medium. The distributed scheduling method is used for scheduling tasks in a big data platform, and the distributed scheduling method comprises the following steps: S1, determining the range...
Gespeichert in:
Hauptverfasser: | , , |
---|---|
Format: | Patent |
Sprache: | chi ; eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | The invention provides a distributed scheduling method and device, electronic equipment and a computer storage medium. The distributed scheduling method is used for scheduling tasks in a big data platform, and the distributed scheduling method comprises the following steps: S1, determining the range of calculation tasks needing to be scheduled, and generating an initial task set; S2, performing layer-by-layer statistics on tasks of upstream reference objects of the tasks in all the initial task sets, and arranging the tasks according to an upstream and downstream sequence; S3, estimating the memory and processor overhead of each task, calculating respective scores through a resource overhead assessment algorithm, and sorting according to the scores to generate an execution task set; and S4, distributing the tasks in the execution task set to a plurality of computing nodes of the big data platform so as to enable the computing nodes to execute respective tasks. According to the distributed scheduling method pr |
---|