Preventing reader starvation during order preserving data stream consumption

The present disclosure involves systems, software, and computer implemented methods for preventing reader starvation in a distributed system. One example method includes receiving, at a node that manages a data stream, a write-intent request from a writer for a first partition, the data stream assoc...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Gruschko, Boris, Foebel, Florian, Strenge, Martin, Mohr, Christian, Friedmann, Bjoern
Format: Patent
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The present disclosure involves systems, software, and computer implemented methods for preventing reader starvation in a distributed system. One example method includes receiving, at a node that manages a data stream, a write-intent request from a writer for a first partition, the data stream associated with a graph that includes levels of partitions including a current level; identifying the first partition for the writer; providing partition information for the first partition to enable the writer to write to the first partition; determining that the writer has previously written to a second partition associated with the current level; in response to determining that the writer has previously written to the second partition: creating a new current level; associating the first partition with the new current level; identifying partitions associated with the current level; and providing a close-level instruction to each local node that includes a partition associated with the current level.