User-Centric Communication With Aerial Network for 6G: A Reinforcement Learning Approach

Meeting the diverse needs of user verticals requires innovative cellular architectures that can offer additional degrees of freedom to provide on-demand services. The terrestrial user-centric radio access network (UC-RAN) stands out as an excellent choice for this purpose. However, a drawback of UC-RAN is its tendency to prioritize high-priority verticals, often resulting in a subpar quality of experience for low-priority verticals. This issue is particularly exacerbated in hotspot areas. To address this problem, we introduce an aerial network integrated with terrestrial UC-RAN to provide coverage to users which are not served by the terrestrial network. Furthermore, we analyze the impact of key configuration and optimization parameters (COPs) such as location, transmit power, altitude and beamwidth of aerial base stations (ABS) on system key performance indicators (KPIs) such as coverage, latency satisfaction, average spectral efficiency and energy efficiency. We formulate a robust multi-objective function to maximize these KPIs without biasing towards any specific KPI(s). Finally, we propose a deep reinforcement learning optimization framework based on the state-of-the-art soft actor-critic algorithm to control ABS COPs and optimize system KPIs. Experimental evaluations demonstrate that the proposed optimization framework can converge to near-optimal solutions derived from the pseudo brute force in a few thousand epochs.