On the integration of crowd knowledge in pattern recognition

•The first work that reveals the key components in an knowledge integration problem.•A performance lower bound of knowledge integration is derived.•The impact of various parameter settings is studied via Monte-Carlo simulations. This paper is concerned with the fundamentals of integrating crowd knowledge such as ratings, opinions or tags provided by the internet users. As a concrete example, we consider the problem of image recognition based on user-provided tags. Each user is assumed to have certain knowledge about the images, which can be incomplete or only of partial relevance to the recognition task. Each user is also assumed to have his own choice of tag vocabulary, possibly different from the set of prescribed labels for image recognition. We argue that a user’s knowledge can be separated into the structure of the knowledge and the representation of the structure (namely, his tag vocabulary). This perspective advocates a systematic three-step methodology for crowd knowledge integration in such applications, whereby the problem of interest is decoupled into three sub-problems in tandem: knowledge structure aggregation, vocabulary interpretation, and label assignment. We derive a lower bound for the achievable error probability. Using this bound and via Monte-Carlo simulations, we investigate the performance of a knowledge integration system in relation to various parameter settings.