Design,implementation and practice of parallel processing system for a large-scale heterogeneous data

With the rapid development of Internet and IoT applications,data processing has gradually expanded from structured to structured,semi-structured and unstructured hybrid heterogeneous data processing mode.A large-scale heterogeneous data parallel processing system was designed.Based on the functional...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:大数据 2020-07, Vol.6, p.2020030-1
Hauptverfasser: Zhengxun XIA, Shengmei LUO, Yuanhao SUN, Jianfei TANG, Yan ZHANG
Format: Artikel
Sprache:chi
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:With the rapid development of Internet and IoT applications,data processing has gradually expanded from structured to structured,semi-structured and unstructured hybrid heterogeneous data processing mode.A large-scale heterogeneous data parallel processing system was designed.Based on the functional view of a unified platform,the unified resource management framework was adopted to store and query a variety of heterogeneous data,including structured,JSON/XML,graph data,document data,etc.Adopting a unified database language,the parallel computing across data types and database engines was realized,and the needs of multi business application development were met.The feasibility of the system is verified by standard evaluation environment and commercial deployment.
ISSN:2096-0271