Background of shape contexts for point matching

•A new kind of features (BSC) is proposed for the point matching problem.•The Coordinate-Gradient Space is proposed to calculate the BSC.•Multiple experiments demonstrate the effectiveness of the proposed method. Point matching seeks the optimal correspondences between two sets of points. It is often tackled via assigning each point a set of features and minimizing the dissimilarities of corresponded features. In this process, a kind of widely applied features is the Shape Contexts (SC). Given a reference point, SC exploits the relative positions between this point and the remaining points. However, SC omits the relative positions among the remaining points, which limits its distinctiveness. In this paper, we propose a new kind of features, called the Background of Shape Contexts (BSC), which encodes not only these relative positions between the reference point and the remaining points, but also those among the remaining points. The BSC concisely presents the relative positions among feature points by edges among the points. The distribution of edges is then calculated in a Coordinate-Gradient Space (CGS). The CGS is a combination of a gradient space and a Euclidean space which can be 2D or 3D. Finally, the BSC exploits histograms to capture the distribution of edges in the CGS. We evaluate the BSC on multiple matching experiments over widely used datasets and demonstrate the effectiveness of the BSC.