Techniques and Architectures for Utilizing a Change Log to Support Incremental Data Changes
Techniques and mechanisms for incremental data ingestion are disclosed. Raw data is received from multiple disparate sources to be consumed in an environment for collecting unformatted raw data. The environment has at least a delta data table and a delta notification table. A write to an entry in th...
Gespeichert in:
Hauptverfasser: | , , , , , , , |
---|---|
Format: | Patent |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | Techniques and mechanisms for incremental data ingestion are disclosed. Raw data is received from multiple disparate sources to be consumed in an environment for collecting unformatted raw data. The environment has at least a delta data table and a delta notification table. A write to an entry in the delta data table is attempted. Entries to the delta data table specify at least records indicating changes to objects in the environment. A write a corresponding entry to the delta notification table is attempted in response to a successful write attempt to the delta data table. The delta notification table entry includes information about delta data table entries for a specified period. At least one data consumer is notified that the delta data table has been modified. |
---|