Unstructured data processing method and device based on Map/Reduce

The invention provides an unstructured data processing method and device based on Map/Reduce and relates to the technical field of cloud computing and distributed data systems. The method includes the steps that object proxies of unstructured data are established in advance; the mapping relation amo...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
1. Verfasser: LUO JINGNING
Format: Patent
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The invention provides an unstructured data processing method and device based on Map/Reduce and relates to the technical field of cloud computing and distributed data systems. The method includes the steps that object proxies of unstructured data are established in advance; the mapping relation among the object proxies, data files and serialized processing nodes is established; a Key-Value pair set based on the object proxies is generated and segmented through a Map/Reduce computing framework so that each computational node can obtain the corresponding Key-Value pair set sequence; by means of the object proxies in the Key-Value pair set and the mapping relation, the serialized processing nodes corresponding to objects are determined; the objects are subjected to dynamic serialized processing through the serialized processing nodes, and data entities of the objects are generated; the data entities of the objects are subjected to deserialized processing through the computational nodes, and the memory objects are reconstructed so that subsequent calculating and processing can be performed on the memory objects.