The recovery of a near optimal layer representation for an entire image sequence

Wang and Adelson see (IEEE Trans. Image Proc., vol.3, no.5, p.625-38, 1994) proposed the layer representation as a convenient representation for video coding, format conversion and special effects. We present a technique to extract the near optimal layer representation of an image sequence using the EM algorithm. We incorporate forward/backward motion estimation to obtain a sharp segmentation in the presence of occlusion and uncovering. We demonstrate that the algorithm is capable of tracking layers within a sequence, and detecting when layers enter or leave the sequence.