System and method for regularizing data between data source and data destination
A system for regularising data transferred between a data source and data destination wherein the data corresponds to a data category and has specific data fields. The system comprises a data fetching module for fetching data from the source 102, a data transformation module for comparing data forma...
Gespeichert in:
Hauptverfasser: | , , , , , , |
---|---|
Format: | Patent |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | A system for regularising data transferred between a data source and data destination wherein the data corresponds to a data category and has specific data fields. The system comprises a data fetching module for fetching data from the source 102, a data transformation module for comparing data formats of values of data fields of the fetched data to pre-defined data formats and transforming the data format to the corresponding pre-defined format based on a determined deviation from said pre-defined format 104-110, a data validation module to receive the pre-defined data formats and either the transformed or fetched data to confirm is the data formats match the pre-defined formats and transmit the data onto the data destination 112-116. The destination is implemented using a database arrangement. The validation module may generate an error log when the data formats do not match. There may be a data regularisation module which receives data from the validation module, determines a variance of the data from the pre-defined format, identifies a solution involving changing the data formats of the data fields and transmitting the resolved data to the transformation module. The data source may be a database. The data fetching module may be a web-crawler. |
---|