Qualitative Spatiotemporal Analysis Using an Oriented Energy Representation

This paper presents an approach to representing and analyzing spatiotemporal information in support of making qualitative, yet semantically meaningful distinctions at the earliest stages of processing. A small set of primitive classes of spatiotemporal structure are proposed that correspond to categories of stationary, coherently moving, incoherently moving, flickering, scintillating and “too unstructured to support further inference”. It is shown how these classes can be represented and distinguished in a uniform fashion in terms of oriented energy signatures. Further, empirical results are presented that illustrate the use of the approach in application to natural imagery. The importance of the described work is twofold: (i) From a theoretical point of view a semantically meaningful decomposition of spatiotemporal information is developed. (ii) From a practical point of view, the developed approach has the potential to impact real world image understanding and analysis applications. As examples: The approach could be used to support early focus of attention and cueing mechanisms that guide subsequent activities by an intelligent agent; the approach could provide the representational substrate for indexing video and other spatiotemporal data.