Real-time job distribution method and system based on streaming query

The invention relates to the technical field of data processing, and provides a real-time job distribution method and system based on streaming query, which fundamentally abandons the dependence on traditional heavy middleware by fusing JDBC streaming query, multi-thread asynchronous production cons...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: CHEN ZHIWEI, LIN JIAMIN, LIN CHEN, SHANGGUAN HUIBAI, CHEN XIFU, LI ZHIHANG
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The invention relates to the technical field of data processing, and provides a real-time job distribution method and system based on streaming query, which fundamentally abandons the dependence on traditional heavy middleware by fusing JDBC streaming query, multi-thread asynchronous production consumption and dynamic cache management technologies. The innovative system can be packaged into a portable jar package, the integration and use process of the innovative system in different projects is greatly simplified, so that the whole working process is simplified, the starting process of development is accelerated, a multi-thread asynchronous processing method is adopted, the data production process and the data consumption process are conducted in parallel, the system load is effectively balanced, and the development efficiency is improved. The data processing efficiency is remarkably improved, and in the multi-thread asynchronous production consumption mode, an interruption and wake-up mechanism is arranged,