Big data merging method based on Bloom filter

The invention discloses a big data merging method based on a Bloom filter. The method comprises the following steps: S1, caching syslog log data sent by different devices/hosts in batches by adopting Redis; s2, the syslog log data cached in batches by the Redis are consumed, and after the consumed syslog log data are subjected to analysis processing, a syslog log data merging field encryption value is obtained; s3, performing screening processing on the syslog log data merging field encryption value by utilizing a bloom filter, and transferring the screened syslog log data to a database; according to the method, the syslog data are cached in batches through Redis to perform merge field encryption value analysis processing, and through merge field processing, a large amount of redundant data in the syslog data is reduced, the storage space is saved, the use cost of the database is reduced, and the use efficiency of the database is improved. 本发明公开了一种基于布隆过滤器的大数据归并方法，包括步骤：S1、采用Redis批量缓存不同设备/主机发送的syslog日志数据；S2、对Re