Design,implementation and practice of parallel processing system for a large-scale heterogeneous data
With the rapid development of Internet and IoT applications,data processing has gradually expanded from structured to structured,semi-structured and unstructured hybrid heterogeneous data processing mode.A large-scale heterogeneous data parallel processing system was designed.Based on the functional...
Gespeichert in:
Veröffentlicht in: | 大数据 2020-07, Vol.6, p.2020030-1 |
---|---|
Hauptverfasser: | , , , , |
Format: | Artikel |
Sprache: | chi |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | With the rapid development of Internet and IoT applications,data processing has gradually expanded from structured to structured,semi-structured and unstructured hybrid heterogeneous data processing mode.A large-scale heterogeneous data parallel processing system was designed.Based on the functional view of a unified platform,the unified resource management framework was adopted to store and query a variety of heterogeneous data,including structured,JSON/XML,graph data,document data,etc.Adopting a unified database language,the parallel computing across data types and database engines was realized,and the needs of multi business application development were met.The feasibility of the system is verified by standard evaluation environment and commercial deployment. |
---|---|
ISSN: | 2096-0271 |