PROCESSING OF RESIDUALS IN VIDEO CODING

According to aspects of the invention there is provided a method of modifying sets of residuals data where residual data can be used to correct or enhance data of a base stream, for example a frame of a video encoded using a legacy video coding technology. According to a first aspect there is provided a method of encoding an input video, the method comprising: receiving an input video comprising a plurality of frames; generating one or more sets of residuals on a frame-by-frame basis based on a difference between a given frame of the input signal and one or more reconstructed versions of the given frame at one or more respective spatial resolutions; selectively modifying the one or more sets of residuals; and encoding the one or more sets of modified residuals to generate one or more respective encoded streams for the input video, wherein the method further comprises: determining a sum of absolute differences pixel metric based on one or more of the plurality of frames, the pixel metric being determined for one or more coding units of the one or more of the plurality of frames; and wherein the one or more sets of residuals are selectively modified based on the pixel metric.