System and method for using low complexity maximum likelihood decoder in a mimo decoder

A method and system for performing quadrature amplitude modulation (QAM) decoding of a received signal includes finding for each layer a region in a first constellation diagram of the received signal, the region including a portion of the first constellation diagram, the portion having the same size of a second constellation diagram, and a first constellation order of the received signal is higher than a second constellation order of the second constellation diagram; and, for each layer: finding a first portion of bits based on bits that are constant among constellation points located in the region of the layer; decoding the received signal using a QAM decoder having the second constellation order to obtain a second portion of bits; adjusting the second portion of bits based on the region of the layer; and merging the first portion of bits with the second portion of bits to obtain a decoded symbol.