Validating an Emulation-Based Cybersecurity Model With a Physical Testbed

For researchers studying cyber-physical system security, working with realistic datasets is essential. To produce the datasets, the existing methodology is to emulate the cyber network. A challenge is that the industrial control systems (ICS) network consists of not just computers and communication equipment, but also field devices that collect data and execute controls. These devices play a significant role in the operation and the security of the system. However, in comparison to the cyber network, the research reproducibility and realism of the cyber-physical system emulation and its data has received far less attention. This article thus develops an approach to answer, "How well can emulated devices replicate the behavior of physical intelligent electronics devices (IEDs) in a realistic cyber attack and defense environment?" To study this, we perform a comparison study based on an emulation experiment using the minimega testbed environment that is entirely virtual and a hardware-in-the-loop experiment using the Resilient Energy Systems Lab ( RESLab ) cyber-physical testbed featuring real industrial controllers and communications devices. Results show that under different reconnaissance attack scenarios, RESLab generates realistic datasets that validate the emulation-based cybersecurity model in minimega . The approach is generalizable toward validating the realism of other types of ICS devices in security studies.