Data real-time cleaning system and cleaning method based on binlog
The invention discloses a data real-time cleaning system and a cleaning method based on binlog. An architecture platform comprises a reference metadata collection unit, the reference metadata collection unit is in electrical output connection with a data processing unit, and the data processing unit...
Gespeichert in:
Hauptverfasser: | , , |
---|---|
Format: | Patent |
Sprache: | chi ; eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | The invention discloses a data real-time cleaning system and a cleaning method based on binlog. An architecture platform comprises a reference metadata collection unit, the reference metadata collection unit is in electrical output connection with a data processing unit, and the data processing unit is in electrical output connection with a reference metadata generation unit. The reference metadata generation unit is in electrical output connection with a reference metadata classification unit, the reference metadata classification unit is in electrical output connection with a metadata database, the metadata database is in electrical output connection with a metadata clearing module, and the metadata clearing module is in electrical output connection with a metadata publishing unit.By obtaining benchmark metadata from MySQL database, the DDL operation information extracted in the process of pulling the binlog is managed in a centralized manner, so that a developer only needs to concentrate on own service dev |
---|