Repartitioning data in a distributed computing system

A method of repartitioning data in a distributed computing system 100 from at least one source 110, 115 to at least one sink 150, 155 through an intermediary transient memory, wherein the source allocates a segment of the intermediary memory 130 to be written to, writes to the segment, then publishe...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Hossein Ahmadi, Matthew B Tolton, Michael Entin
Format: Patent
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:A method of repartitioning data in a distributed computing system 100 from at least one source 110, 115 to at least one sink 150, 155 through an intermediary transient memory, wherein the source allocates a segment of the intermediary memory 130 to be written to, writes to the segment, then publishes the segment to be read by the source. In the embodiments, a shared memory is accessible for write by a plurality of sources and accessible for read by a plurality of sinks. Instead of allocation of memory and publication of address being controlled centrally by the intermediary memory, the allocation and publication of addresses is controlled by the sources in place of a centralised controller. The transient memory may be flushed to permanent memory when it reaches a threshold. The method is suited to remote direct memory access (RDMA) techniques. Computer system and storage device also claimed.