Virtual object overlay onto uncalibrated camera images for augmented reality

We propose a new registration algorithm for overlaying a virtual object generated in a graphics system onto a real‐world image sequence captured by an uncalibrated camera for augmented reality. In our method, a Projective Grid Space (PGS) is defined by using two base images captured from different positions with uncalibrated cameras. In the PGS, correct perspective views of a virtual object can be registered onto an image sequence captured by an uncalibrated camera in real time. Since our method is based on the projective geometry that is defined by two base images, the registration does not require any markers with known 3D positions, but several (at least eight) image feature points are needed for estimating fundamental matrices of the augmented camera with the base images. Based on this method, we constructed an augmented reality system for an uncalibrated camera without any 3D position information in the real world, which runs nearly at video rate (11 frames per second). Experimental results demonstrate the advantages of this method. © 2003 Wiley Periodicals, Inc. Syst Comp Jpn, 34(5): 47–55, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.10261