Enhanced Regular Expression as a DGL for Generation of Synthetic Big Data

Synthetic data generation is generally used in performance evaluation and function tests in data-intensiveapplications, as well as in various areas of data analytics, such as privacy-preserving data publishing (PPDP)and statistical disclosure limit/control. A significant amount of research has been conducted on tools andlanguages for data generation. However, existing tools and languages have been developed for specificpurposes and are unsuitable for other domains. In this article, we propose a regular expression-based datageneration language (DGL) for flexible big data generation. To achieve a general-purpose and powerful DGL,we enhanced the standard regular expressions to support the data domain, type/format inference, sequence andrandom generation, probability distributions, and resource reference. To efficiently implement the proposedlanguage, we propose caching techniques for both the intermediate and database queries. We evaluated theproposed improvement experimentally. KCI Citation Count: 0