DMON: A Distributed Heterogeneous N-Variant System

N-Variant Execution (NVX) systems utilize software diversity techniques for enhancing software security. The general idea is to run multiple different variants of the same program alongside each other while monitoring their run-time behavior. If the internal disparity between the running variants causes observable differences in response to malicious inputs, the monitor can detect such divergences in execution and then raise an alert and/or terminate execution. Existing NVX systems execute multiple, artificially diversified program variants on a single host. This paper presents a novel, distributed NVX design that executes program variants across multiple heterogeneous host computers; our prototype implementation combines an x86-64 host with an ARMv8 host. Our approach greatly increases the level of "internal different-ness" between the simultaneously running variants that can be supported, encompassing different instruction sets, endianness, calling conventions, system call interfaces, and potentially also differences in hardware security features. A major challenge to building such a heterogeneous distributed NVX system is performance. We present solutions to some of the main performance challenges. We evaluate our prototype system implementing these ideas to show that it can provide reasonable performance on a wide range of realistic workloads.