Three-dimensional subband coding with motion compensation

Three-dimensional (3-D) frequency coding is an alternative approach to hybrid coding concepts used in today's standards. The first part of this paper presents a study on concepts for temporal-axis frequency decomposition along the motion trajectory in video sequences. It is shown that, if a two-band split is used, it is possible to overcome the problem of spatial inhomogeneity in the motion vector field (MVF), which occurs at the positions of uncovered and covered areas. In these cases, original pixel values from one frame are placed into the lowpass-band signal, while displaced-frame-difference values are embedded into the highpass band. This technique is applicable with arbitrary MVF's; examples with block-matching and interpolative motion compensation are given. Derivations are first performed for the example of two-tap quadrature mirror filters (QMF's), and then generalized to any linear-phase QMF's. With two-band analysis and synthesis stages arranged as cascade structures, higher resolution frequency decompositions are realizable. In the second part of the paper, encoding of the temporal-axis subband signals is discussed. A parallel filterbank scheme was used for spatial subband decomposition, and adaptive lattice vector quantization was employed to approach the entropy rate of the 3-D subband samples. Coding results suggest that high-motion video sequences can be encoded at significantly lower rates than those achievable with conventional hybrid coders. Main advantages are the high energy compaction capability and the nonrecursive decoder structure. In the conclusion, the scheme is interpreted more generally, viewed as a motion-compensated short-time spectral analysis of video sequences, which can adapt to the quickness of changes. Although a 3-D multiresolution representation of the picture information is produced, a true multiresolution representation of motion information, based on spatio-temporal decimation and interpolation of the MVF, is regarded as the still-missing part.< >