Flume multi-thread test method and system and computer storage medium

The invention provides a Flume multi-thread test method and system and a computer storage medium. The method comprises the steps: creating a message transmission theme, wherein a data sender and a consumer conduct data transmission through the theme; continuously capturing a file, and adding the fil...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
1. Verfasser: ZHU YONGFANG
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The invention provides a Flume multi-thread test method and system and a computer storage medium. The method comprises the steps: creating a message transmission theme, wherein a data sender and a consumer conduct data transmission through the theme; continuously capturing a file, and adding the file content to a target file; transmitting the target file to a multi-thread kafka producer through anagent1 of flume; sinking the target file in the kafka producer to a multi-thread destination through the agent2 of flume; and judging a flume multi-thread transmission function according to the fileinformation received by the destination. The Insight platform Flume component is verified to have a multi-thread transmission function, large-batch data transmission is supported, the test efficiencyis improved, human resources are saved, and a guarantee and a basis are provided for clustered Flume multi-thread test. 本发明提供了一种Flume多线程测试的方法、系统及计算机存储介质,方法包括创建消息传输主题,数据发送者和消费者通过所述主题进行数据传输;持续抓取文件,将所述文件内容追加至目标文件中;通过flume的agent1将目