TY - JOUR

T1 - Advanced Convolutional Neural Networks for Nonlinearity Mitigation in Long-Haul WDM Transmission Systems

AU - Sidelnikov, Oleg

AU - Redyuk, Alexey

AU - Sygletos, Stylianos

AU - Fedoruk, Mikhail

AU - Turitsyn, Sergei K.

N1 - Funding Information: Manuscript received July 4, 2020; revised October 1, 2020 and November 22, 2020; accepted December 22, 2020. Date of publication January 14, 2021; date of current version April 16, 2021. This work was supported in part by the Russian Science Foundation under Grant 17-72-30006, and in part by SS and SKT support by EPSRC Programme Grant TRANSNET (EP/R035342/1). (Corresponding author: Oleg Sidelnikov.) Oleg Sidelnikov is with the Nonlinear Photonics Laboratory, Novosibirsk State University, Novosibirsk 630090, Russia (e-mail: o.sidelnikov@g.nsu.ru). Publisher Copyright: © 2020 IEEE. Copyright: Copyright 2021 Elsevier B.V., All rights reserved.

PY - 2021/4/15

Y1 - 2021/4/15

N2 - Practical implementation of digital signal processing for mitigation of transmission impairments in optical communication systems requires reduction of the complexity of the underlying algorithms. Here, we investigate the application of convolutional neural networks for compensating nonlinear signal distortions in a 3200~km fiber-optic 11x400-Gb/s WDM PDM-16QAM transmission link with a focus on the optimization of the corresponding algorithmic complexity. We propose a design that includes original initialisation of the weights of the layers by a filter predefined through the training a single-layer convolutional neural network. Furthermore, we use an enhanced activation function that takes into account nonlinear interactions between neighbouring symbols. To increase learning efficiency, we apply a layer-wise training scheme followed by joint optimization of all weights applying additional training to all of them together in the large multi-layer network. We examine application of the proposed convolutional neural network for the nonlinearity compensation using only one sample per symbol and evaluate complexity and performance of the proposed technique.

AB - Practical implementation of digital signal processing for mitigation of transmission impairments in optical communication systems requires reduction of the complexity of the underlying algorithms. Here, we investigate the application of convolutional neural networks for compensating nonlinear signal distortions in a 3200~km fiber-optic 11x400-Gb/s WDM PDM-16QAM transmission link with a focus on the optimization of the corresponding algorithmic complexity. We propose a design that includes original initialisation of the weights of the layers by a filter predefined through the training a single-layer convolutional neural network. Furthermore, we use an enhanced activation function that takes into account nonlinear interactions between neighbouring symbols. To increase learning efficiency, we apply a layer-wise training scheme followed by joint optimization of all weights applying additional training to all of them together in the large multi-layer network. We examine application of the proposed convolutional neural network for the nonlinearity compensation using only one sample per symbol and evaluate complexity and performance of the proposed technique.

KW - Complexity theory

KW - Convolution

KW - Convolutional neural networks

KW - Nonlinear optics

KW - Nonlinearity mitigation in fiber-optic links

KW - Optical fiber communication

KW - Optical receivers

KW - Training

KW - nonlinearity mitigation in fiber-optic links

UR - http://www.scopus.com/inward/record.url?scp=85099732588&partnerID=8YFLogxK

UR - https://www.elibrary.ru/item.asp?id=46749510

U2 - 10.1109/JLT.2021.3051609

DO - 10.1109/JLT.2021.3051609

M3 - Article

AN - SCOPUS:85099732588

VL - 39

SP - 2397

EP - 2406

JO - Journal of Lightwave Technology

JF - Journal of Lightwave Technology

SN - 0733-8724

IS - 8

M1 - 9324921

ER -