Probability of Error of Linearly Modulated Signals with Gaussian Cochannel Interference in Maximally Correlated Rayleigh Fading Channels

We evaluate the probability of error of linearly modulated signals, such as phase-shift keying (PSK) and quadrature amplitude modulation (QAM), in the presence of Gaussian cochannel interference (CCI) and Rayleigh fading channels. Specifically, we assume that the fading channel of the CCI is maximally correlated with the fading channel of the signal of interest (SOI). In practical applications, the maximal correlation of the CCI channel with the SOI channel occurs when the CCI is generated at the transmitter, such as the multiuser interference in downlink systems, or when a transparent repeater relays some thermal noise together with the SOI. We analytically evaluate the error probability by using a series expansion of generalized hypergeometric functions. A convenient truncation criterion is also discussed. The proposed theoretical approach favorably compares with alternative approaches, such as numerical integration and Monte Carlo estimation. Among the various applications of the proposed analysis, we illustrate the effect of nonlinear amplifiers in orthogonal frequency-division multiplexing (OFDM) systems, the downlink reception of code-division multiple-access (CDMA) signals, and the outdoor-to-indoor relaying of Global Positioning System (GPS) signals.