Towards Non-I.I.D. image classification: A dataset and baselines

•We investigate the Non-I.I.D problem in image classification and give an index NI to measure the degree of distribution shift.•We construct and release a Non-I.I.D. image dataset called NICO, which makes use of contexts to create various Non-IIDness exibly and consciously.•We propose a novel model CNBB with batch balancing module as a baseline of exploiting CNN for general Non-I.I.D. image classification.•Extensive experiments prove the capacity of NICO and the superiority of CNBB. I.I.D.22I.I.D.: Independent and Identically Distributed hypothesis between training and testing data is the basis of numerous image classification methods. Such property can hardly be guaranteed in practice where the Non-IIDness is common, causing instable performances of these models. In literature, however, the Non-I.I.D.33Non-I.I.D: Non-Independent and Identically Distributed image classification problem is largely understudied. A key reason is lacking of a well-designed dataset to support related research. In this paper, we construct and release a Non-I.I.D. image dataset called NICO44NICO: Non-I.I.D. Image dataset with Contexts, which uses contexts to create Non-IIDness consciously. Compared to other datasets, extended analyses prove NICO can support various Non-I.I.D. situations with sufficient flexibility. Meanwhile, we propose a baseline model with ConvNet structure for General Non-I.I.D. image classification, where distribution of testing data is unknown but different from training data. The experimental results demonstrate that NICO can well support the training of ConvNet model from scratch, and a batch balancing module can help ConvNets to perform better in Non-I.I.D. settings.