High-speed PAM4 transmission using directly modulated laser and artificial neural network nonlinear equaliser

Due to its simplicity, low-cost and small footprint, intensity modulation and direct detection (IM/DD) using directly modulated lasers (DMLs) remains the most widely-adopted optical transmission scheme for short-reach applications. However, when the data rate increases to 100 Gb/s and beyond, the DML nonlinearity and fibre chromatic dispersion become major factors that limit system performance. In this paper, we show that artificial neural networks (ANNs) are an effective nonlinear equaliser for enhancing the transmission performance of high-speed IM/DD systems using DML. More specifically, for 56 Gbaud PAM-4 transmission system, a low-complexity ANN equaliser with only 1 hidden layer and 10 neurons can increase the receiver sensitivity at KP-4 FEC limit in comparison with FFE-15 by approximately 1 dB at dispersion of −60 ps/nm and ＋20 ps/nm. In addition, we also show that ANN equaliser outperforms the commonly used Volterra equaliser for 56 Gbaud PAM-4 transmissions. •Low-complexity ANN equaliser for 56 Gbaud PAM4 DML-based fiber transmissions is reported.•ANN equaliser having up to 20 neurons is compared with second order Volterra equaliser with up to 15 taps.•10-neuron ANN equaliser increases sensitivity at KP-4 FEC by ∼1 dB at 20 ps/nm and −60 ps/nm of dispersion.•Once ANN has been trained, complexity of ANN equaliser is much lower than that of Volterra’s equaliser.