ENFORCING DATA SECURITY CONSTRAINTS IN A DATA PIPELINE

A computer-implemented method enforces data security constraints in a data pipeline. The data pipeline takes one or more source datasets as input and performs one or more data transformations on them. The method includes using data defining one or more data security constraints to configure the data pipeline to perform a data transformation on a restricted subset of entries of the source datasets. The restriction is defined by the data defining one or more data security constraints. The method further includes performing the data transformation according to the configuration to produce one or more transformed datasets. The method further includes using the data defining one or more data security constraints to perform a verification on one or more of the transformed datasets to ensure that entries in the one or more of the transformed datasets are restricted as defined by the one or more data security constraints.