Data processing method and system based on data lake
The invention discloses a data processing method and system based on a data lake, which is applied to a platform comprising a data warehouse, and comprises the following steps: classifying all source data information based on a data access specification, constructing a source pasting table on a sour...
Gespeichert in:
Hauptverfasser: | , , , , , , , |
---|---|
Format: | Patent |
Sprache: | chi ; eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | The invention discloses a data processing method and system based on a data lake, which is applied to a platform comprising a data warehouse, and comprises the following steps: classifying all source data information based on a data access specification, constructing a source pasting table on a source pasting layer, and importing a data source file into the data lake; the method comprises the following steps: analyzing a service demand according to a service application, carrying out dimension modeling based on the service demand, creating a dimension table and a fact table, setting a data index according to the fact table, and establishing a market special topic at a market layer based on the data index; checking the field needing to be monitored of the dimension modeling from the source pasting table; constructing a summary table in a summary layer, and performing metadata acquisition and monitoring on the dimension table, the fact table and the summary table; and opening the data tables in the summary laye |
---|