Systems and Methods for Creating Modular Data Processing Pipelines
Systems, methods, and apparatuses are described herein that allow users to create and manage flexible, highly modular data processing pipelines. Such pipelines may be associated with any number of connected nodes connected via dependency injection to define the location and type of data that a pipel...
Gespeichert in:
Hauptverfasser: | , , , |
---|---|
Format: | Patent |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | Systems, methods, and apparatuses are described herein that allow users to create and manage flexible, highly modular data processing pipelines. Such pipelines may be associated with any number of connected nodes connected via dependency injection to define the location and type of data that a pipeline uses as input or output and the operations to be performed by the pipeline. The pipelines may also be associated with context information, which specifies dataset-specific configurations and includes logic required to generate and execute the associated nodes. The context information may further include logic that allows for node substitution, caching of node output, data filtering, and/or dynamic node modification. |
---|