Optimized blind equalization for probabilistically shaped high-order QAM signals

Probabilistically shaped(PS) high-order quadrature amplitude modulation(QAM) signals are attractive to coherent optical communication due to increased spectral efficiency. However, standard digital signal processing algorithms are not optimal to demodulate PS high-order QAM signals. Therefore, a compromise equalization is indispensable to compensate the residual distortion. Meanwhile, the performance of conventional blind equalization highly depends on the accurate amplitude radius and distribution of the signals. The PS high-order QAM signals make the issue worsen because of indistinct amplitude distributions. In this work, we proposed an optimized blind equalization by utilizing a peak-density K-means clustering algorithm to accurately track the amplitude radius and distribution. We experimentally demonstrated the proposed method in a PS 256-QAM coherent optical transmission system and achieved approximately 1 d B optical signal-to-noise ratio improvement at the bit error rate of 1 × 10.