AUTOMATIC DATA LINTING RULES FOR ETL PIPELINES
In the present disclosure, systems and methods are described for allowing a non-code user to create to transform a database in an ETL pipleline. Specifically, as disclosed herein, a user can take a database and receive a ruleset to apply to the database in an ETL pipeline. The data linting system ma...
Gespeichert in:
Hauptverfasser: | , , , , , , |
---|---|
Format: | Patent |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | In the present disclosure, systems and methods are described for allowing a non-code user to create to transform a database in an ETL pipleline. Specifically, as disclosed herein, a user can take a database and receive a ruleset to apply to the database in an ETL pipeline. The data linting system may take the database and extract a schema and a data sample from it. Further, the data linting system may use the schema and data sample to create two rulesets. With these rulesets, the data linting system combines them to create a final ruleset which may be validated using the data sample. The data linting system then sends the final ruleset and the validation report to the user. With this system, the user only needs to give it a database and will receive a ruleset that is able to be immediately used in an ETL pipeline. |
---|