Wireless MapReduce Arrays for Coded Distributed Computing
We consider a wireless distributed computing system based on the MapReduce framework, which consists of three phases: \textit{Map}, \textit{Shuffle}, and \textit{Reduce}. The system consists of a set of distributed nodes assigned to compute arbitrary output functions depending on a file library. The...
Gespeichert in:
Hauptverfasser: | , , |
---|---|
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | We consider a wireless distributed computing system based on the MapReduce
framework, which consists of three phases: \textit{Map}, \textit{Shuffle}, and
\textit{Reduce}. The system consists of a set of distributed nodes assigned to
compute arbitrary output functions depending on a file library. The computation
of the output functions is decomposed into Map and Reduce functions, and the
Shuffle phase, which involves the data exchange, links the two. In our model,
the Shuffle phase communication happens over a full-duplex wireless
interference channel. For this setting, a coded wireless MapReduce distributed
computing scheme exists in the literature, achieving optimal performance under
one-shot linear schemes. However, the scheme requires the number of input files
to be very large, growing exponentially with the number of nodes. We present
schemes that require the number of files to be in the order of the number of
nodes and achieve the same performance as the existing scheme. The schemes are
obtained by designing a structure called wireless MapReduce array that
succinctly represents all three phases in a single array. The wireless
MapReduce arrays can also be obtained from the extended placement delivery
arrays known for multi-antenna coded caching schemes. |
---|---|
DOI: | 10.48550/arxiv.2406.15791 |