Integration of online deep reinforcement learning and federated learning: A dynamic energy management strategy for industrial applications

In the context of federated learning, this paper focuses on managing inherent dynamics and uncertainties, and optimizing energy management for devices in real‐world industrial environments. The problem is formulated by proposing an online deep reinforcement learning algorithm that optimizes model iteration updates between clients and servers, and an energy harvesting strategy that enhances device performance and extends lifespan. Through these innovative methods, solutions are provided for the dynamics and uncertainties in federated learning and a new approach is devised for energy usage optimization and task scheduling. Numerical results demonstrate the efficiency of the approach in transitioning federated learning from theory to practical application, and offering diversified application scenarios. In summary, this research contributes to the development of federated learning by providing novel insights and methodologies, and highlights the challenges and opportunities for future studies in the field. This paper delves into these issues and presents several innovative research outcomes. We initially propose a novel online deep reinforcement learning algorithm that efficiently tackles the dynamics and uncertainties in federated learning by optimizing model iteration updates between clients and servers.