Load balancing in a distributed processing system

Methods, systems, and apparatus are described for load balancing in a distributed processing system. In one aspect, a method includes monitoring, for each data processor in a distributed processing system, a rate of cursor progress for the data processor based on timestamps of data units that have b...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Chernyak Vyacheslav Alekseyevich, Bekiroglu Kaya
Format: Patent
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Methods, systems, and apparatus are described for load balancing in a distributed processing system. In one aspect, a method includes monitoring, for each data processor in a distributed processing system, a rate of cursor progress for the data processor based on timestamps of data units that have been processed, wherein the rate of cursor progress for each data processor specifies a rate of change of an oldest timestamp of an oldest data unit being processed by the data processor; determining a progress threshold for the distributed processing system based on the respective rates of cursor progress of the data processors; determining, based on a first rate of cursor progress for a first data processor, that the first rate of cursor progress does not meet the progress threshold; and in response to the determination, performing one or more load balancing operations on the distributed processing system.