System and method for real time 2D to 3D conversion of video in a digital camera

Embodiments are directed towards enabling digital cameras to digitally process captured two dimensional image sequences at a real time video rate, to convert the two dimensional image sequences into stereoscopic three dimensional image sequences. In one embodiment, using a pipelining architecture, various statistics are obtained for a captured two dimensional sequence. The statistics are used to estimate depth within each frame image within the sequence, in real time. Using the depth data a disparity map is generated that is provided to a warping component to generate a second perspective of the two dimensional image. The two images for the frame provide a three dimensional perspective for the frame within the sequence. Together the two perspective images for the frame are provided to a video encoder component to encode the stereoscopic three dimensional frame for the sequence.