Sample data/parameter transmission method in big data distributed calculation process

The invention provides a sample data/parameter transmission method in a big data distributed calculation process. The invention relates to a sample data/parameter transmission method in a big data distributed computing process and a computer readable storage medium related to a big data distributed...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: ZHANG XIANGYU, SUN JUNHUAN, LI YANG, ZHANG MANNI
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The invention provides a sample data/parameter transmission method in a big data distributed calculation process. The invention relates to a sample data/parameter transmission method in a big data distributed computing process and a computer readable storage medium related to a big data distributed computing system based on the sample data/parameter transmission method. An efficient RDMA channel is provided for sample data/parameters between a task scheduling engine and a task calculation engine or between task calculation engines by providing efficient and reusable RDMA channel connection and the like, and a completion notification message is transmitted through a TCP channel after transmission is completed, so that the efficiency of the distributed calculation system is further improved. 本发明提供一种大数据分布式计算过程中样本数据/参数的传递方法。其中涉及的大数据分布式计算过程中样本数据/参数的传递方法,以及基于此的大数据分布式计算系统相关的计算机可读存储介质,通过提供高效可复用的RDMA通道连接等为任务调度引擎与任务计算引擎或任务计算引擎与任务计算引擎间的样本数据/参数提供高效的RDMA通道,以及在传输完成后又有通过TCP通道传输完成通知消息以进一步提高分布式计算系统的效率。