Data processing method and device based on flink framework
The invention discloses a data processing method and device based on an flink framework, and relates to the technical field of computers. A specific embodiment of the method comprises the following steps: after a window in an flink framework receives data, calling a method for processing the data in...
Gespeichert in:
1. Verfasser: | |
---|---|
Format: | Patent |
Sprache: | chi ; eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | The invention discloses a data processing method and device based on an flink framework, and relates to the technical field of computers. A specific embodiment of the method comprises the following steps: after a window in an flink framework receives data, calling a method for processing the data in a trigger, and obtaining a trigger state of a current trigger calculation operation; if the trigger state is a trigger state, calling a service processing logic from a window operator to process the data, and summarizing a current processing result into a historical processing result to obtain a summarized result; and after reaching the time of the timer, transmitting the summarized result to a downstream operator based on a predetermined mode. According to the embodiment, the data processing operation and the summarizing result sending operation are separated and triggered respectively, the situation that in a big data scene, a large amount of data are overstocked in a window within a short time, and the storage |
---|