High Spectral Efficiency WDM Transmission Based on Hybrid Probabilistically and Geometrically Shaped 256QAM

We experimentally demonstrated a 10-channel 50-GHz-grid wavelength division multiplexing (WDM) signal transmission over 400-km standard single-mode fiber (SSMF) with the net bit rate of 614.4 Gbps/carrier and the spectral efficiency (SE) of 12.288 b/s/Hz, which is the highest SE for 256QAM to the best of our knowledge. Thanks to the joint pre-equalization and Volterra nonlinear equalization (VNLE), 49-Gbaud polarization division multiplexing (PDM) 256-ary quadrature amplitude modulation (256QAM) signal transmission with a novel constellation using probabilistic shaping (PS) and geometric shaping (GS) techniques is realized, considering the 0.828-normalized generalized mutual information (NGMI) low-density parity-check (LDPC) threshold with 25% overhead. Compared with the regular 256QAM at the same net bit rate, the designed hybrid PS/GS 256QAM at the expense of 1-Gbaud increase in the symbol rate obtains 1.7-dB and 1.5-dB shaping gain in single-channel case and WDM case, respectively. Meanwhile, an improvement of at least 0.6-dB in optical signal to noise ratio (OSNR) over the hybrid PS/GS 256QAM is achieved in comparison with PS-256QAM for both single-channel and WDM transmission.