Age of Information Minimization for UAV-Assisted Internet of Things Networks: A Safe Actor-Critic With Policy Distillation Approach

Thanks to smart manufacturing and artificial intelligence technologies, unmanned aerial vehicles (UAVs) are envisioned to play a critical role in future Internet of things (IoT) networks to execute data collection tasks. In this article, we leverage age of information (AoI) to measure the freshness of data packets received by the UAV from IoT sensors. Considering the heterogeneity of IoT devices, we aim to minimize the weighted sum AoI by jointly optimizing the UAV's trajectory and IoT devices association in UAV-assisted IoT networks, where the UAV's cumulative propulsion energy cost is limited by the battery capacity. Since the optimization object is confined by a set of short-term constraints and a long-term constraint, this problem is modeled as a constrained Markov decision process (CMDP). We leverage safe actor-critic (Safe-AC) to solve the CMDP. To satisfy the mixed constraints, the safe policy set of Safe-AC is induced by a Lyapunov function, thereafter, a policy distillation technology is leveraged to search the optimal policy. Experimental results indicate that our proposed scheme can strictly satisfy the propulsion energy cost budget requirement at the expense of around 2% loss of the reward compared to baseline methods.