A dynamic and scalable parallel Network Intrusion Detection System using intelligent rule ordering and Network Function Virtualization

A Network Intrusion Detection System (NIDS) is a fundamental security tool. However, under heavy network traffic, a NIDS might become a bottleneck. In an overloaded state, incoming and outgoing packets in the network might suffer from long delays since previous packets are still being inspected, and eventually the NIDS starts to drop packets when it runs out of hardware resources. Although many solutions have been suggested in the literature to counter this problem, they are not completely reliable as each of them has limitations. This paper investigates the design of a lightweight elastic architecture which allows parallel processing in an existing NIDS while maintaining the filtering integrity. Furthermore, we propose two adaptive algorithms which dynamically adjust and divide the signature rules evenly across NIDS nodes using a node level parallelism method in order to achieve intelligent rule ordering. We test our approaches in real-life settings by implementing a functioning prototype involving different modern networking technologies. The prototype presented is a Network Function Virtualization (NFV) of an intrusion detection system which utilizes Open vSwitch and Docker containers running Snort in order to provide an elastic system. To the best of our knowledge, there has been no work that orchestrates both scaling and rule splitting and re-ordering of IDS signatures as a part of a holistic elastic IDS solution. The results of this study show that the proposed algorithms are able to equally split the IDS workload and thereby enabling the system to scale by adjusting the number of virtual components which analyse the network traffic. At the same time the experiments indicate that the algorithms can be tuned by a single parameter in order to avoid that some packets go unexamined while simultaneously craving a minimum of the dynamically available computer resources. •A novel design of a parallel NIDS based on NFV and rule distribution is presented.•A modified parallel technique, NL-FP-1, dynamically adjusts and divides signatures.•Prototype experiments demonstrate dynamic adaption to the CPU load on the system.