Video Segmentation with Spatio-Temporal Tubes

Long-term temporal interactions among objects are an important cue for video understanding. To capture such object relations, we propose a novel method for spatio-temporal video segmentation based on dense trajectory clustering that is also effective when objects articulate. We use superpixels of homogeneous size jointly with optical flow information to ease the matching of regions from one frame to another. Our second main contribution is a hierarchical fusion algorithm that yields segmentation information available at multiple linked scales. We test the algorithm on several videos from the web showing a large variety of difficulties. 2013 10th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) , 27 Aug 2013, 30 Aug 2013,