Flume multi-thread test method and system and computer storage medium
The invention provides a Flume multi-thread test method and system and a computer storage medium. The method comprises the steps: creating a message transmission theme, wherein a data sender and a consumer conduct data transmission through the theme; continuously capturing a file, and adding the fil...
Gespeichert in:
1. Verfasser: | |
---|---|
Format: | Patent |
Sprache: | chi ; eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | The invention provides a Flume multi-thread test method and system and a computer storage medium. The method comprises the steps: creating a message transmission theme, wherein a data sender and a consumer conduct data transmission through the theme; continuously capturing a file, and adding the file content to a target file; transmitting the target file to a multi-thread kafka producer through anagent1 of flume; sinking the target file in the kafka producer to a multi-thread destination through the agent2 of flume; and judging a flume multi-thread transmission function according to the fileinformation received by the destination. The Insight platform Flume component is verified to have a multi-thread transmission function, large-batch data transmission is supported, the test efficiencyis improved, human resources are saved, and a guarantee and a basis are provided for clustered Flume multi-thread test.
本发明提供了一种Flume多线程测试的方法、系统及计算机存储介质,方法包括创建消息传输主题,数据发送者和消费者通过所述主题进行数据传输;持续抓取文件,将所述文件内容追加至目标文件中;通过flume的agent1将目 |
---|