A method for deduplicating master-slave synchronization data of a MySQL database

The invention discloses a method for deduplicating master-slave synchronous data of a MySQL database. The method distinguishes the MySQL database synchronization operation from the data deduplicationoperation, and is executed as two continuous operation steps, namely, after the data synchronization operation is completed, duplicate data is found by comparing the synchronized data with the data inthe synchronization log, and the duplicate data is deleted from the library. The method of the invention does not need to perform locking operation on the database table, does not affect the parallelservice function provided by the database, and does not introduce excessive performance overhead, so the method is superior to the existing solution in the aspects of database operation parallelism and performance overhead. 本发明公开种MySQL数据库主从同步数据去重方法。所述方法将MySQL数据库同步操作与数据去重操作区分开来，作为两个连续的操作步骤来执行，即在完成数据同步操作后，通过将本次同步的数据与同步日志中的数据进行比对，找到重复数据，并将从库中的重复数据删除。本发明所述方法无需对数据库表执行加锁操作，不影响数据库对外提供的并行服务功能，同时也不会引入过大的性能开销，因此在数据库操作并行性和性能开销方面优于现有的