Data processing method and device based on flink framework

The invention discloses a data processing method and device based on an flink framework, and relates to the technical field of computers. A specific embodiment of the method comprises the following steps: after a window in an flink framework receives data, calling a method for processing the data in...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
1. Verfasser: CAO JIANQI
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The invention discloses a data processing method and device based on an flink framework, and relates to the technical field of computers. A specific embodiment of the method comprises the following steps: after a window in an flink framework receives data, calling a method for processing the data in a trigger, and obtaining a trigger state of a current trigger calculation operation; if the trigger state is a trigger state, calling a service processing logic from a window operator to process the data, and summarizing a current processing result into a historical processing result to obtain a summarized result; and after reaching the time of the timer, transmitting the summarized result to a downstream operator based on a predetermined mode. According to the embodiment, the data processing operation and the summarizing result sending operation are separated and triggered respectively, the situation that in a big data scene, a large amount of data are overstocked in a window within a short time, and the storage