Q-Learning Based Interference-Aware Channel Handoff for Partially Observable Cognitive Radio Ad Hoc Networks

Channel handoff is a crucial function for Cognitive radio ad hoc networks（CRAHNs）. The absence of centralized infrastructures and the limited power make the handoff design more challenging. A learningbased interference-aware handoff scheme is proposed for distributed CRAHNs. We model the channel handoff process as a Partially observable Markov decision process（POMDP） and adopt a Q-learning algorithm to find an optimal handoff strategy in a long term. The proposed algorithm obtains an efficient transmission performance by considering the interferences among SUs and PUs. To achieve PU awareness, the handoff scheme predicts the PU activities by using the historical channel usage statistics. In addition, we also propose a refined channel selection rule to compromise between learning speed and cumulative transmission reward. The simulation results show that the proposed handoff scheme can adapt to the PU activities and achieves a better performance in terms of high throughput and low collisions. And the learning process keeps a considerable balance between convergence time and cumulative reward.