FAST TRAINING FOR A DATA PIPELINE MONITORING SYSTEM
Various embodiments comprise systems and methods to determine output attributes of a data pipeline. In some examples, a data pipeline monitoring system retrieves historical data generated by a data pipeline and determines generation dates for the historical outputs. The system identifies one or more...
Gespeichert in:
1. Verfasser: | |
---|---|
Format: | Patent |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | Various embodiments comprise systems and methods to determine output attributes of a data pipeline. In some examples, a data pipeline monitoring system retrieves historical data generated by a data pipeline and determines generation dates for the historical outputs. The system identifies one or more attributes of the historical data outputs. The system generates an output model that indicates expected output attributes based on the identified attributes of the historical outputs. The system generates an error threshold based on the model and applies the error threshold to outputs generated by the data pipeline. The system generates alerts when the outputs trigger the error threshold. |
---|