Load balancing in a distributed processing system
Methods, systems, and apparatus are described for load balancing in a distributed processing system. In one aspect, a method includes monitoring, for each data processor in a distributed processing system, a rate of cursor progress for the data processor based on timestamps of data units that have b...
Gespeichert in:
Hauptverfasser: | , |
---|---|
Format: | Patent |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | Methods, systems, and apparatus are described for load balancing in a distributed processing system. In one aspect, a method includes monitoring, for each data processor in a distributed processing system, a rate of cursor progress for the data processor based on timestamps of data units that have been processed, wherein the rate of cursor progress for each data processor specifies a rate of change of an oldest timestamp of an oldest data unit being processed by the data processor; determining a progress threshold for the distributed processing system based on the respective rates of cursor progress of the data processors; determining, based on a first rate of cursor progress for a first data processor, that the first rate of cursor progress does not meet the progress threshold; and in response to the determination, performing one or more load balancing operations on the distributed processing system. |
---|