Dynamic clustering of software defined network switches and controller placement using deep reinforcement learning

Software defined networking (SDN) has emerged as a promising alternative to the traditional networks, offering many advantages, including flexibility in network management, network programmability and guaranteeing application Quality-of-Service (QoS) requirements. In SDN, the control plane is separated from the data plane, and deployed as a logically centralized controller. However, due to the large scale of networks as well as latency and reliability requirements, it is necessary to deploy multiple controllers to satisfy these requirements. The distributed deployment of SDN controllers unveiled new challenges in terms of determining the number of controllers needed, their locations and the assignment of switches to controllers that minimizes flow set delay. In this context, we propose, in this paper, a new method that dynamically computes the optimal number of controllers, determines their optimal locations, and at the same time partitions the set of data plane switches into clusters and assigns them to these controllers. First, we mathematically formulate the controller placement as an optimization problem, whose objectives are to minimize the controller response time, that is the delay between the SDN controller and assigned switches, the Control Load (CL), the Intra-Cluster Delay (ICD) and the Intra-Cluster Throughput (ICT). Second, we propose a simple yet computationally efficient heuristic, called Deep Q-Network based Dynamic Clustering and Placement (DDCP), that leverages the potential of reinforcement and deep learning techniques to solve the aforementioned optimization problem. Experimental results using ONOS controller show that the proposed approach can significantly improve the network performances in terms of response time and resource utilization.