Video data processing system

An apparatus and method for encoding frames of a source frame sequence of video image data includes circuitry 8 encoding source frames using reference frames 13 (e.g. downscaled), 14 (e.g. full resolution), and monitoring circuitry 15 configured to, when the encoding circuitry 8 is encoding a source frame using reference frame(s) 13, 14, monitor memory bandwidth being used. In response to the monitored memory bandwidth being greater than a threshold, the encoding circuitry is controlled to encode a subsequent part of the source frame using a modified video encoding process to restrict the memory bandwidth usage when using one or more reference frames 13, 14 when encoding a subsequent (remaining) part of a source frame. The threshold may be based on a target memory bandwidth used for encoding the whole source frame. A memory bandwidth measure may be based on data read from a cache and/or into the cache from a memory when using reference frames; a moving average or accumulated total of cache/memory accesses may be assessed. Reference blocks may be used. The modified encoding process may restrict the number of reference frames used, frame search area size, or search position.