Intelligent Spectrum and Airspace Resource Management for Urban Air Mobility Using Deep Reinforcement Learning

In an era dominated by a surge in air travel and heightened reliance on efficient communication systems, there's a critical need to smartly allocate frequency resources for aviation communications to efficiently manage airspace operations. This is essential to ensure safe, smooth, and technologically advanced flight services. Over time, techniques for managing frequency resources and new radio technologies have evolved to cope with the increased demands on the system due to growing airspace activities. With the development of Urban Air Mobility (UAM) operations, a fresh challenge has emerged, further burdening the already limited aviation spectrum. There is a pressing need for a new approach to efficiently manage and utilize frequencies. This paper explores the application of Multi-agent Reinforcement Learning (MARL) technique to minimize aircraft mission completion time and enhance safety, all while dealing with the limitations of airspace and frequency resources. The proposed MARL approach utilizes the Value Decomposition Network (VDN) technique to optimize frequency use, flight time, and departure wait times by managing spectrum allocation, vehicle departure, and flight speed. To achieve the goal of minimizing mission completion time, the Markov Decision Process (MDP) is utilized. It takes into account factors like frequency channel availability, signal-to-interference-plus-noise power ratio, aircraft location, and flight status. In our investigation, we develop a case study scenario and assess the performance of the MARL technique through simulation in a hypothetical UAM scenario. The solution is evaluated against Q-Mixing (QMIX), Orthogonal Multiple Access and a Heuristic Greedy Algorithm.