Spark performance prediction method and device, model training method and device, equipment and medium
The invention relates to the technical field of Spark performance prediction, and provides a Spark performance prediction method and device, a model training method and device, equipment and a medium, and the method comprises the steps: collecting historical data of an application program running in...
Gespeichert in:
Hauptverfasser: | , , , , , , |
---|---|
Format: | Patent |
Sprache: | chi ; eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | The invention relates to the technical field of Spark performance prediction, and provides a Spark performance prediction method and device, a model training method and device, equipment and a medium, and the method comprises the steps: collecting historical data of an application program running in a Spark system; preprocessing the historical data to obtain a processed data set; the processed data set is adopted to train a Spark performance prediction model; wherein the Spark performance prediction model comprises a multi-head attention network and a convolutional neural network, and the multi-head attention network is used for extracting semantic information of input data and operation features; the convolutional neural network is used for extracting spatial correlation features of the input data and the job features; predicting performance indexes of application program operation in the Spark system according to the semantic information and the spatial correlation characteristics, and outputting a predicti |
---|