Label shift conditioned hybrid querying for deep active learning

Active learning aims to interactively select the most informative data for accelerating the learning procedure. In this paper, we propose a novel and principled deep active learning approach under label shift—the labeled dataset has a different class distribution with respect to the unlabeled data pool. By formulating the querying procedure as a distribution matching problem, we derive the generalization guarantee under label shift. Moreover, the theoretical results inspire a unified practical framework for deep batch active learning: in the training stage, we jointly correct the label shift and align the semantic conditional distributions; as for the querying stage, label-shift-conditioned hybrid querying (LSCHQ) strategy is proposed to balance uncertainty and diversity under label shift. Empirical results show that our proposed method is highly effective and outperforms other baseline algorithms over a variety of benchmarks and a real-world dataset. •Theoretical analysis for deep active learning under label shift.•Label shift correction and conditional distribution alignment during training.•Querying strategy considers uncertainty, diversity and the affect of label shift.