Consumption information flow processing method and system for Apache Kafka

The invention discloses a consumption information flow processing method and system for Apache Kafka. The method comprises the following steps that: the processing time of data pulling is calculated,whether the processing time is greater than pulling overtime time or not is judged, and if the processing time is greater than the pulling overtime time, new data are pulled and processed; and if theprocessing time is not greater than pulling overtime time, the pause time of a data pulling thread is calculated, and the data pulling is paused within the pause time. With the method adopted, a Javathread pool can be utilized to the maximum extent; program running efficiency is improved; the frequency of triggering an Apache Kafka election mechanism is reduced; and the throughput of an information system is improved. 本发明公开了一种用于Apache Kafka的消费信息流处理方法及系统，方法包括：计算拉取数据的处理时间，判断处理时间是否大于拉取超时时间，若是，则：拉取新的数据进行处理；若否，则：计算拉取数据线程暂停时间，在暂停时间内暂停拉取数据。本发明能够最大限度的利用Java线程池，提高程序运行效率，减少触发Apache Kafka选举机制的次数，进而提升信息系统的吞吐量。