Multilevel Coding for Nonlinear ISI Channels

This paper proposes a new nonlinear equalization approach for the transmission of high-density constellations over nonlinear ISI channels. Multilevel coding is performed at the transmitter, and multistage equalization and decoding at the receiver. Two equalizers are introduced. One uses nonlinear intersymbol interference (ISI) cancellation and has linear complexity in the channel memory L, and the number of decoding stages K. The other employs the reduced-state Bahl-Cocke-Jelinek-Raviv (BCJR) algorithm, which shows better performance and complexity on the order of K . 2(L). Capacity analysis and code simulation results show that these two schemes can achieve significant improvement over the conventional approaches that view nonlinear ISI as additive noise.