Data processing method and apparatus

The present invention discloses a data processing method and apparatus. When mainframes are moved down to an open platform, batch tasks are received. The method comprises: enabling a master node that receives a batch task to determine a plurality of member nodes for processing the batch tasks from n...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: YAN CHEN, MIN GUILONG
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The present invention discloses a data processing method and apparatus. When mainframes are moved down to an open platform, batch tasks are received. The method comprises: enabling a master node that receives a batch task to determine a plurality of member nodes for processing the batch tasks from nodes connected to the master node; according to information of the plurality of member nodes, segmenting table data corresponding to the batch tasks, so as to obtain a plurality of block data; correspondingly sending the block data to the plurality of member nodes; acquiring a plurality of result data sent by the plurality of member nodes; and combining the plurality of result data to obtain a processing result of the batch tasks. The method disclosed by the present invention solves the problem that when the open platform processes batch processing data, the bottleneck of the throughput will result in lengthening of task time, which further results in failure in completing the batch task within a specific time wind