Resilient Massive Access for SAGIN: A Deep Reinforcement Learning Approach

In the visionary ideals of "Internet of Everything" and "Digital Twins", the future 6G will deeply integrate diverse heterogeneous networks such as satellite and aerial networks to support seamless connectivity and efficient interoperability, also known as space-air-ground integrated networks (SAGIN), in which the grant-free uplink random access based on Slotted ALOHA (S-ALOHA) can reduce access latency and complexity for massive Internet of Things (IoT) devices. However, with the increasing number of IoT users, the collision probability of S-ALOHA escalates and further degrades the system performance. In this paper, we focus on the massive IoT device uplink access in SAGIN aided by high altitude platform stations (HAPS), investigating power allocation for IoT devices to maximize system access capability and spectral efficiency (SE). Specifically, we first optimize 3D deployment of HAPS. Then the resilient massive access (RMA) based on flexible fusion of S-ALOHA and non-orthogonal multiple access methods is proposed. To maximize system SE with device power constraints, we model the sequential decision problem as a Markov decision process and solve it with the Advantage Actor-Critic (A2C) algorithm. Simulation results demonstrate the proposed RMA can significantly improve the IoT terminal successful access probability and the resource scheduling based on A2C also significantly increases the system SE with low complexity.