Apparatus and method for frequency lattice modulation in wireless communication system

The disclosure relates to a pre-5th-Generation (5G) or 5G communication system to be provided for supporting higher data rates Beyond 4th-Generation (4G) communication system such as long term evolution (LTE). According to various embodiments, a transmission device in a wireless environment may include a transceiver, and a processor operatively coupled to the transceiver. The processor may be configured to generate a codeword related to transmission data, generate symbols from the codeword by using a plurality of transmission symbol vectors determined using a linear combination of vectors corresponding to respective rows of a lattice generation matrix, and transmit a signal generated based on the symbols via the transceiver.