Federated Deep Reinforcement Learning for Task Offloading in MEC-enabled Heterogeneous Networks

The integration of mobile edge computing (MEC) and heterogeneous networks enables network operators to provide task offloading services to a large number of user devices (UDs) for low-latency task processing by equipping macro base stations and densely-deployed small base stations with edge servers. Federated deep reinforcement learning allows each UD to collaboratively learn useful knowledge from the interaction with the environment in a privacy-preserving and high-efficiency way and thus has been applied to solve the task offloading problem in recent studies. However, very few of these studies have considered the energy and time costs incurred by the federated learning process. In this paper, the goal is to minimize the total UDs' energy consumption while guaranteeing deadline constraints considering both the task offloading process and the federated learning process in MEC-enabled heterogeneous networks. Towards this end, we propose a federated deep Q-network (DQN) method where each UD optimizes the offloading decision for the offloading process and the participation decision and training volume for the learning process based on its local DQN model. The simulation results demonstrate the proposed method is superior to several existing methods in terms of energy efficiency and quality of service.