Dynamic projection mapping on deformable stretchable materials using boundary tracking

We present the first method for dynamic projection mapping on rectangular, deformable and stretchable elastic materials using consumer-grade time of flight (ToF) depth cameras. We use a B-Spline patch to model the projection surface without explicitly modeling the deformation. Leveraging the nature of deformable and stretchable materials, we propose an efficient tracking method that can track the boundary of the surface material. This achieves realistic mapping even in the interior of the surface such that the projection appears to be printed on the material. The surface representation is updated in real-time using GPU based computations. Further, we also show that the speed of these updates is limited by the speed of the camera and therefore can be adopted for higher speed cameras as well. [Display omitted] •Exploit properties of rational B-spline patches to model the projection surface.•Track surface boundary using simple markers or marker-less depth features.•Model stretches and constrained projection mapping of elastic materials.•Using consumer-grade projector–camera hardware for real-time performance.