Data real-time cleaning system and cleaning method based on binlog

The invention discloses a data real-time cleaning system and a cleaning method based on binlog. An architecture platform comprises a reference metadata collection unit, the reference metadata collection unit is in electrical output connection with a data processing unit, and the data processing unit...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: SU SHENGBIN, JIANG MENGJUAN, WU GUIFENG
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The invention discloses a data real-time cleaning system and a cleaning method based on binlog. An architecture platform comprises a reference metadata collection unit, the reference metadata collection unit is in electrical output connection with a data processing unit, and the data processing unit is in electrical output connection with a reference metadata generation unit. The reference metadata generation unit is in electrical output connection with a reference metadata classification unit, the reference metadata classification unit is in electrical output connection with a metadata database, the metadata database is in electrical output connection with a metadata clearing module, and the metadata clearing module is in electrical output connection with a metadata publishing unit.By obtaining benchmark metadata from MySQL database, the DDL operation information extracted in the process of pulling the binlog is managed in a centralized manner, so that a developer only needs to concentrate on own service dev