Demonstration of 200 Gbps PAM-4 transmission in a limited-bandwidth system using a two-tap THP with nonlinearity compensator

In this paper, we have proposed a digital signal processing (DSP) method for a 200 Gbps pulse-amplitude modulation level-4 (PAM-4) transmission in a bandwidth-limited system. To mitigate inter-symbol interferences (ISI) in the system and reduce the propagation of errors in the receiver, the Tomlinson-Harashima precoding (THP) technique was applied to the transmitter. For simplicity, the number of THP taps was limited to two (i.e., one coefficient to be optimized); the residual ISI resulting from the short tap was compensated for with a decision feedback equalizer (DFE). The performance of the proposed DSP was analyzed via experiments and simulations. Compared to conventional DSP techniques (such as DFE and the maximum likelihood sequence estimation (MLSE) without THP), the proposed DSP performs better within the limited-bandwidth region. As a result, the soft-decision forward error correction (SD-FEC) threshold (of 2 x 10-2) was satisfied even when the bandwidth fell to below 35% of the baud rate. To enhance this performance, post-filter and MLSE were additionally applied, after which its performance was investigated. In addition, the nonlinear characteristics of the components were alleviated by applying a pre-distortion technique to the THP using look-up tables; this further enhanced performance, lowering the SD-FEC threshold's bandwidth to 32% of the baud rate.