A Contextual MAB-Based Two-Timescale Scheme for RIS-Assisted Systems

In this paper, we propose a contextual multi-armed bandits (CMAB)-based two-timescale (TTS) scheme for the reconfigurable intelligent surface (RIS)-assisted massive MIMO systems. Different from the existing TTS schemes which requires many coherence times for estimating the channel covariance matrix (CCM), the proposed scheme utilizes the bandit learning to learn the CCM from historical transmission data for the phase shift design, which avoids spending multiple coherence times estimating the CCM, and thereby improving spectral efficiency. Particularly, we formulate the problem of the phase shift design at the RIS side as a CMAB problem in the large timescale, where the phase shift is considered as the context and the reward is related to the CCM of the cascaded channel. During the learning process, the orthogonal matching pursuit-based algorithm is used to learn the CCM, and an improved ε-greedy algorithm is proposed to design the phase shift. In the small timescale, the combining vector is designed to mitigate interference. Simulation results validate the high spectral efficiency of the proposed TTS scheme.