Design of Limited Buffer Size for Storing Multiple Reference Frames in a Video Codec

Multiple reference frames are recommended for motion estimation and compensation to provide coding efficiency and error concealment in video codecs. However, multiple reference frames take more hardware cost than one reference frame used in the conventional video codecs. Hence, there is a need to reduce the memory size for storing multi-frames. In this work, we look closely at a MacroBlock (MB) of the currently decoded frame to determine the data type, compressed or reconstructed one, for storing itself, according to the coding parameters of blocks in this MB. In addition, the computational complexity and memory-required distribution (CMD) of currently decoded frame is explored in a group of picture (GOP). According to CMD, the computational complexity is minimized by means of making a full use of the given memory size. Furthermore, when the given memory size is insufficient, the signal concealment scheme is utilized to enables MBs being stored as near-end motion vectors without residuals. Experimental results reveal that the proposed scheme can effectively constrain the memory usage under a given memory size for storing five reference frames as full as possible. In addition, because adequately allocating the buffer size to store the currently decoded frame, a fairly good video quality can be achieved as compared to the conventional work using re-compression schemes for storing reference frames.