A realistic evaluation of indoor robot position tracking systems: The IPIN 2016 competition experience

•The IPIN 2016 Track 4 competition focused on indoor mobile robot positioning.•Location estimation of an industrial mobile robot tracking a continuous trajectory.•Trajectory geometrically described and unknown for participants.•The evaluation procedure allows the comparison of different technologies. We report a novel open competition aimed at evaluating accurate robot position tracking in indoor environments. The competition was organized within the IPIN 2016 (Indoor Positioning and Indoor Navigation international Conference). Here, we describe the competition, the competitors and their final results. The challenges of this new competition included: tracking an industrial robot following an unknown path but with a defined ground-truth, and open positioning system to be deployed on-site, with no restrictions apart from those related to safety issues. Our aim here is to provide sufficient detail to serve as a solid basis for future competition initiatives with a similar scope, using common metrics and objective evaluation procedures. In addition, the real systems evaluated represent state-of-the-art performance, and thus offer interesting solutions to the problem posed in the competition.