Encrypted traffic classification based on Gaussian mixture models and Hidden Markov Models

To protect user privacy (e.g., IP address and sensitive data in a packet), many traffic protection methods, like traffic obfuscation and encryption technologies, are introduced. However, these methods have been used by attackers to transmit malicious traffic, posing a serious threat to network security. To enhance network traffic supervision, this paper proposes a new traffic classification model based on Gaussian mixture models and hidden Markov models, named MGHMM. To evaluate the effectiveness of the proposed model, we first classify protocols and identify the obfuscated traffic by experiments. Then, we compare the classification performance of MGHMM with that of the latest Vector Quantiser-based traffic classification algorithm. On the basis of the experiment, the relation between the classification and the number of hidden Markov states, and the number of mixture of Gaussian distributions required to describe the hidden states, are analyzed. •Only need inter-packet time and packet size for traffic classification.•Analyze the discrete distribution and timing pattern of the flow features.•Perform well in traffic classification at multiple traffic levels.•Obtain the best classification results with minimal resource overhead.