Spectral efficiency limits of pre-filtered modulation formats

We investigate the spectral efficiency (SE) limit of pre-filtered modulation in optical fiber communication systems. We show that pre-filtering induced symbol correlation generates a modulation with memory and, thus, a higher SE limit than that of the original memoryless modulation. The SE limits of a series of modulation formats with varying number L of correlated symbols producing 50% of the original bandwidth are evaluated, which approach the SE limit of a modulation format with 50% pre-filtering as L -> infinity. We show that the SE limit of a modulation format with L correlated symbols approximates a lower bound for the SE limit of the corresponding pre-filtered format. (C) 2010 Optical Society of America