Parallel K-means optimization method based on Spark and ASPSO

The invention provides a parallel K-means optimization method based on Spark and ASPSO, and the method comprises the following steps: S1, roughly dividing a data set through a segmentation function, calculating a Pearson's correlation coefficient and a correlation coefficient threshold value of...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: MAO YIMIN, LIAO LIEFA, GAN DEJIN
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The invention provides a parallel K-means optimization method based on Spark and ASPSO, and the method comprises the following steps: S1, roughly dividing a data set through a segmentation function, calculating a Pearson's correlation coefficient and a correlation coefficient threshold value of a data grid through employing a grid division strategy PCCV, and dividing the data grid to obtain grid units; S2, adopting an SPFG strategy to carry out local area coverage on the data points, updating sample points in the data set, forming an area cluster, and obtaining the cluster number of local clustering; S3, adopting an ASPSO strategy, calculating adaptive parameters, and obtaining a local cluster centroid; S4, calculating the cluster radius of each cluster by adopting a CRNN strategy, performing similarity judgment according to a similarity function of the clusters, and combining the clusters with large similarity by combining a Spark parallel computing framework; and S5, outputting a clustering result. Accordin