Track deformable objects from point clouds with structure preserved registration

Manipulating deformable objects is a challenging task for robots. The major difficulty lies in how to track these objects accurately, robustly, and efficiently, considering they have infinite-dimensional configuration space. To deal with these problems, this paper proposes a novel state estimator to track deformable objects from point clouds. A non-rigid registration method, named structure preserved registration (SPR), is developed to update the estimation by registering the object model towards the current point cloud measurement. Both the local structure and the global topology of the deformable object are considered during registration, which improves the estimation robustness under noise, outliers, and occlusions. The tracking result is further refined by running a dynamic simulation in parallel, which enforces the estimates to satisfy the physical constraints of the object. A series of real-time tracking experiments on 1D objects (ropes) and 2D objects (clothes) are performed to evaluate the proposed state estimator. A wire harness manipulation platform is also introduced where robots can manipulate soft wires to desired shapes and autonomously evaluate the manipulation quality through visual feedback.