Top-down model fitting for hand pose recovery in sequences of depth images

State-of-the-art approaches on hand pose estimation from depth images have reported promising results under quite controlled considerations. In this paper we propose a two-step pipeline for recovering the hand pose from a sequence of depth images. The pipeline has been designed to deal with images taken from any viewpoint and exhibiting a high degree of finger occlusion. In a first step we initialize the hand pose using a part-based model, fitting a set of hand components in the depth images. In a second step we consider temporal data and estimate the parameters of a trained bilinear model consisting of shape and trajectory bases. We evaluate our approach on a new created synthetic hand dataset along with NYU and MSRA real datasets. Results demonstrate that the proposed method outperforms the most recent pose recovering approaches, including those based on CNNs. [Display omitted] •A top-down strategy for hand pose recovery in depth images proposed•Firstly, nearest shapes are extracted based on a new shape descriptor.•Hand fingers are segmented and palm is extracted based on kNN shapes.•Finger models are fitted to the hand given palm and finger segments.•A previously trained bilinear temporal model is fitted to refine occluded joints.