Bayesian generative adversarial network emulator based end-to-end learning strategy of the probabilistic shaping for OAM mode division multiplexing IM/DD transmission

Orbital angular momentum (OAM) mode division multiplexing (MDM) has emerged as a new multiplexing technology that can significantly increase transmission capacity. In addition, probabilistic shaping (PS) is a well-established technique that can increase the transmission capacity of an optical fiber to close to the Shannon limit. However, both the mode coupling and the nonlinear impairment lead to a considerable gap between the OAM-MDM channel and the conventional additive white Gaussian noise (AWGN) channel, meaning that existing PS technology is not suitable for an OAM-MDM intensity-modulation direct-detection (IM-DD) system. In this paper, we propose a Bayesian generative adversarial network (BGAN) emulator based on an end-to-end (E2E) learning strategy with probabilistic shaping (PS) for an OAM-MDM IM/DD transmission with two modes. The weights and biases of the BGAN emulator are treated as a probability distribution, which can be accurately matched to the stochastic nonlinear model of OAM-MDM. Furthermore, a BGAN emulator based on an E2E learning strategy is proposed to find the optimal probability distribution of PS for an OAM-MDM IM/DD system. An experiment was conducted on a 200 Gbit/s two OAM modes carrierless amplitude phase-32(CAP-32) signal over a 5 km ring-core fiber transmission, and the results showed that the proposed BGAN emulator outperformed a conventional CGAN emulator, with improvements in modelling accuracy of 29.3% and 26.3% for the two OAM modes, respectively. Moreover, the generalized mutual information (GMI) of the proposed E2E learning strategy outperformed the conventional MB distribution and the CGAN emulator by 0.31 and 0.33 bits/symbol and 0.16 and 0.2 bits/symbol for the two OAM modes, respectively. Our experimental results demonstrate that the proposed E2E learning strategy with the BGAN emulator is a promising candidate for OAM-MDM IM/DD optic fiber communication.