Decoding of arbitrarily ordered and interleaved macroblocks

Presented herein is a system and method for decoding flexibly ordered macroblocks. Slice groups are received, wherein each slice group includes one or more slices, and each slice contains any number of encoded macroblocks. The macroblocks forming a slice group are not necessarily spatially contiguous with respect to raster scan order. Each macroblock is associated with a particular context which serially provides each macroblock to a processor for decoding. The decoder selectively decodes the macroblock from the first context or second context, depending on which context includes the next macroblock in raster scan order.