Priority-Aware Reinforcement-Learning-Based Integrated Design of Networking and Control for Industrial Internet of Things

Industrial Internet of Things (IIoT) envisions the tight coupling of numerous critical industrial manufacturing subsystems, such as control, networking, and computing through the ubiquitous Internet of Things technologies. Nonetheless, such interconnectivity poses significant challenges to the successful management and operation of massively distributed industrial manufacturing systems. Without carefully integrated system design, the nonoptimal management and operation of highly intertwined subsystems can lead to the loss of productivity and ultimately the value of factories and plants. To address this issue, in this article, we conduct the integrated design that is capable of simultaneously configuring both control and networking subsystems in IIoT with consideration for their inherent interdependencies. We first analyze the performance of the dynamic backoff exponential (BE) in IEEE 802.15.4 carrier-sense multiple access (CSMA) and show the performance improvement of dynamic BE. We then design a model-free reinforcement learning algorithm to configure the control and networking subsystems automatically via systematic trial and error, as it is impractical to build a model for a highly intertwined complex IIoT system. Considering the time-sensitive characteristics of IIoT systems, we design priority-aware policies based on importance among networking traffic (i.e., sensing traffic and actuation traffic) to improve the convergence speed. The experimental results demonstrate that our priority-aware reinforcement-learning-based integrated design can successfully reconfigure the complex and highly intertwined IIoT system at runtime with a minimal convergence time. Besides, our approach reduces convergence time by 37.5%, and energy consumption by 9.2%, compared to the standard reinforcement learning approach.