Network intrusion detection system based on recursive feature addition and bigram technique

Network and Internet security is a critical universal issue. The increased rate of cyber terrorism has put national security under risk. In addition, Internet attacks have caused severe damages to different sectors (i.e., individuals, economy, enterprises, organizations and governments). Network Intrusion Detection Systems (NIDS) are one of the solutions against these attacks. However, NIDS always need to improve their performance in terms of increasing the accuracy and decreasing false alarms. Integrating feature selection with intrusion detection has shown to be a successful approach since feature selection can help in selecting the most informative features from the entire set of features. Usually, for the stealthy and low profile attacks (zero – day attacks), there are few neatly concealed packets distributed over a long period of time to mislead firewalls and NIDS. Besides, there are many features extracted from those packets, which may make some machine learning-based feature selection methods to suffer from overfitting especially when the data have large numbers of features and relatively small numbers of examples. In this paper, we are proposing a NIDS based on a feature selection method called Recursive Feature Addition (RFA) and bigram technique. The system has been designed, implemented and tested. We tested the model on the ISCX 2012 data set, which is one of the most well-known and recent data sets for intrusion detection purposes. Furthermore, we are proposing a bigram technique to encode payload string features into a useful representation that can be used in feature selection. In addition, we propose a new evaluation metric called (combined) that combines accuracy, detection rate and false alarm rate in a way that helps in comparing different systems and selecting the best among them. The designed feature selection-based system has shown a noticeable improvement on the performance using different metrics.