Long-range spatiotemporal motion understanding using spatiotemporal flow curves

A spatiotemporal (ST) image cube, created by stacking a temporally dense sequence of images together, is a temporally coherent data representation. Using ST surface flow, i.e., the extension of optical flow to ST surfaces, it is shown how ST flow curves can be recovered and then used to detect groups of flow curves such that each group represents a single object or surface in the scene undergoing motion. The algorithm forms clusters of similar flow curves and is based on constraints called the temporal uniqueness constraints. First, a point in an image can only move to at most one point in the next image. Second, a point in an image can come from at most one point in the previous image. When these constraints are violated, or it appears that they are violated, occlusion or disocclusion has occurred and therefore can also be detected. Successful grouping of coherent regions of the ST cube for two gray-level image sequences is shown.< >