Method and system for realizing high availability of Spark Thrift Server
The invention relates to the technical field of big data storage and calculation, and particularly provides a method for realizing high availability of a Spark Thrift Server, which comprises the following steps of: when the Spark Thrift Server is started, acquiring a ZooKeeper client connection, and...
Gespeichert in:
Hauptverfasser: | , , , , |
---|---|
Format: | Patent |
Sprache: | chi ; eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | The invention relates to the technical field of big data storage and calculation, and particularly provides a method for realizing high availability of a Spark Thrift Server, which comprises the following steps of: when the Spark Thrift Server is started, acquiring a ZooKeeper client connection, and writing Spark Thrift Server information into the ZooKeeper; the method comprises the following steps: when a client is connected with a Spark Thrift Server service, firstly requesting a ZooKeeper service and acquiring all Spark Thrift Server nodes in a namespace during connection, electing one Spark Thrift Server node through an equalization algorithm, connecting to the Spark Thrift Server, and when the Spark Thrift Server service stops, cancelling node information from the ZooKeeper at the same time. Compared with the prior art, the method has the advantages that when a certain Spark Thrift Server node breaks down, the client side is not affected, and therefore high availability is achieved.
本发明涉及大数据存储与计算技术领域,具体提 |
---|