Real-time job distribution method and system based on streaming query
The invention relates to the technical field of data processing, and provides a real-time job distribution method and system based on streaming query, which fundamentally abandons the dependence on traditional heavy middleware by fusing JDBC streaming query, multi-thread asynchronous production cons...
Gespeichert in:
Hauptverfasser: | , , , , , |
---|---|
Format: | Patent |
Sprache: | chi ; eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | The invention relates to the technical field of data processing, and provides a real-time job distribution method and system based on streaming query, which fundamentally abandons the dependence on traditional heavy middleware by fusing JDBC streaming query, multi-thread asynchronous production consumption and dynamic cache management technologies. The innovative system can be packaged into a portable jar package, the integration and use process of the innovative system in different projects is greatly simplified, so that the whole working process is simplified, the starting process of development is accelerated, a multi-thread asynchronous processing method is adopted, the data production process and the data consumption process are conducted in parallel, the system load is effectively balanced, and the development efficiency is improved. The data processing efficiency is remarkably improved, and in the multi-thread asynchronous production consumption mode, an interruption and wake-up mechanism is arranged, |
---|