Data shuffling method and device

The embodiment of the invention discloses a data shuffling method and device. A specific embodiment of the method comprises the following steps: acquiring multiple stages; writing data into a continuous reading module based on remote procedure call data shuffling in the multi-stage stage from a cont...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: ZHU LIANGCHANG, LIU LINHONG, HUANG TENGFEI
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The embodiment of the invention discloses a data shuffling method and device. A specific embodiment of the method comprises the following steps: acquiring multiple stages; writing data into a continuous reading module based on remote procedure call data shuffling in the multi-stage stage from a continuous writing module based on remote procedure call data shuffling in the multi-stage stage; aligning data by a continuous read module based on remote procedure call data shuffling in multiple stages. According to the embodiment, the data shuffling operation in the continuous processing mode is realized through remote procedure call, the data transmission rate in the data shuffling stage is improved, and the data transmission delay is reduced. 本申请实施例公开了数据洗牌方法和装置。该方法的一具体实施方式包括:获取多级阶段;从多级阶段中的基于远程过程调用数据洗牌的持续写模块向多级阶段中的基于远程过程调用数据洗牌的持续读模块写入数据;通过多级阶段中的基于远程过程调用数据洗牌的持续读模块将数据对齐。该实施方式通过远程过程调用实现了在连续处理模式下的数据洗牌操作,提高了数据洗牌阶段数据的传输速率,降低了数据的传输时延。