Structured Reinforcement Learning for Delay-Optimal Data Transmission in Dense mmWave Networks

We study the data packet transmission problem ( mmDPT ) in dense cell-free millimeter wave (mmWave) networks, i.e., users sending data packet requests to access points (APs) via uplinks and APs transmitting requested data packets to users via downlinks. Our objective is to minimize the average delay...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on wireless communications 2024-10, Vol.23 (10), p.14546-14559
Hauptverfasser: Wang, Shufan, Xiong, Guojun, Zhang, Shichen, Zeng, Huacheng, Li, Jian, Panwar, Shivendra S.
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:We study the data packet transmission problem ( mmDPT ) in dense cell-free millimeter wave (mmWave) networks, i.e., users sending data packet requests to access points (APs) via uplinks and APs transmitting requested data packets to users via downlinks. Our objective is to minimize the average delay in the system due to APs' limited service capacity and unreliable wireless channels between APs and users. This problem can be formulated as a restless multi-armed bandits problem with fairness constraint ( RMAB-F ). Since finding the optimal policy for RMAB-F is intractable, existing learning algorithms are computationally expensive and not suitable for practical dynamic dense mmWave networks. In this paper, we propose a structured reinforcement learning (RL) solution for mmDPT by exploiting the inherent structure encoded in RMAB-F . To achieve this, we first design a low-complexity and provably asymptotically optimal index policy for RMAB-F . Then, we leverage this structure information to develop a structured RL algorithm called mmDPT-TS , which provably achieves an \tilde {\mathcal {O}}(\sqrt {T}) Bayesian regret. More importantly, mmDPT-TS is computation-efficient and thus amenable to practical implementation, as it fully exploits the structure of index policy for making decisions. Extensive emulation based on data collected in realistic mmWave networks demonstrate significant gains of mmDPT-TS over existing approaches.
ISSN:1536-1276
1558-2248
DOI:10.1109/TWC.2024.3416437