An efficient and scalable architecture for real-time distortion removal and rectification of live camera images

Non-linear image distortions are a serious problem for computer vision and industrial real-time image processing systems. This paper describes and analyzes an efficient architecture to perform non-linear image transformations in real-time at a latency of just a few image lines. It can, for example, be used to correct image distortions caused by wide angle lenses and to rectify images within stereo vision systems. The architecture was inspired by Charles Babbage's Difference Engine and is based on finite differences in conjunction with polynomial approximations. The required degree of the polynomials and number of significant register bits can be configured arbitrarily to achieve the required accuracy at low hardware costs. The system has been implemented and tested within a Spartan-3E FPGA. It was evaluated for the practical application of a stereo camera head and hand tracking system for gesture based interaction. For this configuration fourth degree polynomials were sufficient to guarantee a maximum error of less than 0.5 pixels.