Cluster zooming-based Spark configuration parameter automatic adjustment and optimization method

The invention discloses a cluster zooming-based Spark configuration parameter automatic adjustment and optimization method. The method comprises the steps of (1) establishing a cluster; (2) selectinga configuration parameter set; (3) determining configuration parameter value types and ranges; (4) zooming the cluster; (5) training a random forest model; (6) screening an optimal configuration; and(7) verifying a configuration effect. The method can be applied to the technical field of massive data processing; by zooming the memory configuration parameter value ranges and a to-be-processed dataquantity of a distributed memory computing framework Spark, the time for evaluating each configuration is shortened; the relationships between the configurations and the influence of the cluster performance of the distributed memory computing framework Spark are established through the random forest model; and the configuration for optimizing the cluster performance of the distributed memory computing framework Spark consi