Estimation of optical flow based on higher-order spatiotemporal derivatives in interlaced and non-interlaced image sequences

This contribution 1 1 Partial results have been presented in a preliminary version at the ECCV '94 [42]. investigates local differential techniques for estimating optical flow and its derivatives based on the brightness change constraint. By using the tensor calculus representation we build the Taylor expansion of the gray-value derivatives as well as of the optical flow in a spatiotemporal neighborhood. Such a formulation provides a unifying framework for all existing local differential approaches and allows to derive new systems of equations for the estimation of the optical flow and of its derivatives. We also tested various optical flow estimation approaches on real image sequences recorded by a calibrated camera which was fixed on the arm of a robot. By moving the arm of the robot along a precisely defined trajectory, we can determine the true displacement rate of scene surface elements projected into the image plane and compare it quantitatively with the results of different optical flow estimators. Since the optical flow estimators are based on gray-value derivatives of up to fourth-order, we were forced to develop modified Gaussian derivative filters to obtain acceptable estimates for the derivatives. Further, we show quantitatively that these filters contribute to a much more robust optical flow estimation. In addition, successive lines of TV-cameras have an offset in time due to the interlace technique. We demonstrate the adaptation of filter kernels for estimating higher-order spatiotemporal derivatives in interlaced image sequences.