Data quality inspection method and system based on distributed big data computing platform
The invention relates to a data quality inspection method and system based on a distributed big data computing platform, and belongs to the technical field of data processing. According to the method, newly-added or re-reported file handling data on the day is obtained in an incremental extraction m...
Gespeichert in:
1. Verfasser: | |
---|---|
Format: | Patent |
Sprache: | chi ; eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | The invention relates to a data quality inspection method and system based on a distributed big data computing platform, and belongs to the technical field of data processing. According to the method, newly-added or re-reported file handling data on the day is obtained in an incremental extraction mode, the obtained newly-added or re-reported file handling data is associated and combined with the historical file handling data stored in the platform, the updated file handling data is obtained, and quality inspection is conducted on the updated file handling data. The whole process is not limited by reporting time and data volume, multiple times of reporting and batch reporting of historical data can be realized, and when the data is not changed completely, the data is filtered and does not need to be reported again, so that the reporting pressure is reduced, the operation efficiency and computing power of quality inspection are greatly improved, and the problems of low qualified rate and low timeliness of the |
---|