Adjusting direct memory access transfers used in video decoding

An apparatus having a first memory and a circuit is disclosed. The first memory may be configured to store a list having a plurality of read requests. The read requests generally (i) correspond to a plurality of blocks of a reference picture and (ii) are used to decode a current picture in a bitstream carrying video. The circuit may be configured to (i) rearrange the read requests in the list based on at least one of (a) a size of a buffer in a second memory and (b) a width of a data bus of the second memory and (ii) copy a portion of the reference picture from the second memory to a third memory using one or more direct memory access transfers in response to the list.