Method and system for realizing high availability of Spark Thrift Server

The invention relates to the technical field of big data storage and calculation, and particularly provides a method for realizing high availability of a Spark Thrift Server, which comprises the following steps of: when the Spark Thrift Server is started, acquiring a ZooKeeper client connection, and...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: ZHOU YONGJIN, XIA CHUANTAO, LI GUOTAO, QIN XUEJIAO, WEI ZHENG
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The invention relates to the technical field of big data storage and calculation, and particularly provides a method for realizing high availability of a Spark Thrift Server, which comprises the following steps of: when the Spark Thrift Server is started, acquiring a ZooKeeper client connection, and writing Spark Thrift Server information into the ZooKeeper; the method comprises the following steps: when a client is connected with a Spark Thrift Server service, firstly requesting a ZooKeeper service and acquiring all Spark Thrift Server nodes in a namespace during connection, electing one Spark Thrift Server node through an equalization algorithm, connecting to the Spark Thrift Server, and when the Spark Thrift Server service stops, cancelling node information from the ZooKeeper at the same time. Compared with the prior art, the method has the advantages that when a certain Spark Thrift Server node breaks down, the client side is not affected, and therefore high availability is achieved. 本发明涉及大数据存储与计算技术领域,具体提