Abstract -We analyze the performance of fractionally spaced maximum likelihood sequence estimation (MLSE) equalizers in OOK and PAM4 optical systems using optical and electrical components with cut-off frequencies less than the baud rate. It has been demonstrated that signals suffering from optical and electrical impairments can be efficiently equalized in cheap direct-detection optical receivers using MLSE equalizers with one or two samples, depending on the extent of bandwidth limitations.Keywords -Maximum likelihood sequence estimation, chromatic dispersion, bandlimited systems, direct detection, digital receiver.
I. INTRODUCTIONOHERENT optical systems using both polarizations and high-level modulation formats enable data transmission at much higher bit rates compared with noncoherent systems [1]. Most noncoherent systems are based on intensity modulation with direct detection (IM-DD) also known as on/off keying (OOK) and non-return to zero (NRZ). These systems suffer from chromatic dispersion (CD) and polarization mode dispersion (PMD), and their deployment was limited to rates up to 10-Gb/s. Transponders for 40-Gb/s systems use slightly more complex modulation, such as optical duobinary (ODB) and differential phase shift keying (DPSK), and increasingly crowding out 10-Gb/s systems due to increased traffic demands. Further improvements have been achieved by noncoherent 40-Gb/s DQPSK transponders that also suffer from fiber impairments and require expensive external PMD compensators. Most commercial optical links contain dispersion compensation fibers (DCF) and CD has become uncritical for noncoherent systems.Direct detection systems are very attractive because of price, power consumption, size, and so on. In IM-DD 10-Gb/s systems, the received signal can be equalized by several techniques such as feed-forward (FFE), decision feedback (DFE), and MLSE equalizers. FFE equalizers are the least efficient solution while DFE can extend optical Paper received April 2, 2015; revised September 18, 2015; accepted September 20, 2015 reach but suffers from error multiplication at a low optical signal-to-noise ratio (OSNR). Additionally, DFE equalizers experience implementation problems at high bit rates. The MLSE technique is the best one, enabling the longest reach and best performance at the price of very complex digital signal processing (DSP). However, IC technology advances make complex MLSE equalizers feasible. For example, the power consumption of a 64-state MLSE equalizer realized in 28nm technology for 28-Gb/s binary systems is approximately 1.5 W.The Viterbi algorithm significantly reduces finding the most likely transmitted data sequence and enables the realization of practical MLSE decoders [2]. Besides their usage in convolutional decoders and many other applications, MLSE equalizers can also be used in direct detection optical receivers for compensating intersymbol interference caused by CD and PMD [3][4][5][6]. Commercial 10-Gb/s MLSE receivers and DSP chips including the Viterbi decoder with 4, 8 or 16 ...