Convergence analysis of an augmented algorithm for fully complex-valued neural networks

Xu, Dongpo; Zhang, Huisheng; Mandic, Danilo P.

doi:10.1016/j.neunet.2015.05.003

Cited by 32 publications

(15 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We can also apply some techniques that are specific to the complex domain. For example [34], inspired by the theory of widely linear adaptive filters, augments the input to the CVNN with its complex conjugate x * . Additional improvements can be obtained by replacing the real-valued µ with a complex-valued learning rate [35], which can speed up convergence in some scenarios.…”

Section: B Complex-valued Neural Networkmentioning

confidence: 99%

Complex-Valued Neural Networks With Nonparametric Activation Functions

Scardapane

Vaerenbergh

Hussain

et al. 2020

IEEE Trans. Emerg. Top. Comput. Intell.

View full text Add to dashboard Cite

Complex-valued neural networks (CVNNs) are a powerful modeling tool for domains where data can be naturally interpreted in terms of complex numbers. However, several analytical properties of the complex domain (e.g., holomorphicity) make the design of CVNNs a more challenging task than their real counterpart. In this paper, we consider the problem of flexible activation functions (AFs) in the complex domain, i.e., AFs endowed with sufficient degrees of freedom to adapt their shape given the training data. While this problem has received considerable attention in the real case, a very limited literature exists for CVNNs, where most activation functions are generally developed in a split fashion (i.e., by considering the real and imaginary parts of the activation separately) or with simple phase-amplitude techniques. Leveraging over the recently proposed kernel activation functions (KAFs), and related advances in the design of complex-valued kernels, we propose the first fully complex, non-parametric activation function for CVNNs, which is based on a kernel expansion with a fixed dictionary that can be implemented efficiently on vectorized hardware. Several experiments on common use cases, including prediction and channel equalization, validate our proposal when compared to real-valued neural networks and CVNNs with fixed activation functions.

show abstract

Section: B Complex-valued Neural Networkmentioning

confidence: 99%

Complex-Valued Neural Networks With Nonparametric Activation Functions

Scardapane

Vaerenbergh

Hussain

et al. 2020

IEEE Trans. Emerg. Top. Comput. Intell.

View full text Add to dashboard Cite

show abstract

“…We should mention that, as shown by Lemma 4.2 in [29] and Lemma 2 in [30], the Lipschitz condition in Theorem 3.1 and the value L do exist for a variety of activation functions, including both locally analytic functions and traditional splitcomplex functions. From (11) it is important to point out that, although the real part of the stepsize must be positive, there is no sign restriction for the imaginary part.…”

Section: Convergence Analysismentioning

confidence: 94%

Is a Complex-Valued Stepsize Advantageous in Complex-Valued Gradient Learning Algorithms?

Zhang

Mandic

2016

IEEE Trans. Neural Netw. Learning Syst.

Self Cite

View full text Add to dashboard Cite

Abstract-Complex gradient methods have been widely used in learning theory, and typically aim to optimize real-valued functions of complex variables. The stepsize of complex gradient learning methods (CGLM) is a positive number, and little is known about how a complex stepsize would affect the learning process. To this end, we undertake a comprehensive analysis of CGLMs with complex stepsize, including the search space, convergence properties and the dynamics near critical points. Furthermore, several adaptive stepsizes are derived by extending the Barzilai-Borwein method to the complex domain, in order to show that the complex stepsize is superior to the corresponding real one in approximating the information in the Hessian. A numerical example is presented to support the analysis.

show abstract

“…. , x(n − N + 1)] T contains the first N elements in x(n), x b (n) in (38) can be represented in a widely nonlinear form in x c (n) (or equivalently, in x(n)) as [44], [45] x…”

Section: Proposed Augmented Nonlinear Lms Based Si Canceller and mentioning

confidence: 99%

An Augmented Nonlinear LMS for Digital Self-Interference Cancellation in Full-Duplex Direct-Conversion Transceivers

Zhe¹,

Xia²,

Pei³

et al. 2018

IEEE Trans. Signal Process.

Self Cite

View full text Add to dashboard Cite

In future full-duplex communications, the cancellation of self-interference (SI) arising from hardware non-idealities will play an important role in the design of mobile-scale devices. To this end, we introduce an optimal digital SI cancellation solution for shared-antenna-based direct-conversion transceivers. To establish that the underlying widely linear signal model is not adequate for strong transmit signals, the impact of various circuit imperfections, including power amplifier (PA) distortion, frequency-dependent I/Q imbalance, quantization noise and thermal noise, on the performance of the conventional augmented least mean square (LMS) based SI canceller, is analyzed. In order to achieve a sufficient signal-to-interference-plus-noise ratio (SINR) when the nonlinear SI components are not negligible, we propose an augmented nonlinear LMS based SI canceller for a joint cancellation of both the linear and nonlinear SI components by virtue of a widely nonlinear model fit. A rigorous mean and mean square performance evaluation is conducted to justify the performance advantages of the proposed scheme over the conventional augmented LMS solution. Simulations on orthogonal frequency division multiplexing (OFDM)-based wireless local area network (WLAN) standard compliant waveforms support the analysis.

show abstract

Convergence analysis of an augmented algorithm for fully complex-valued neural networks

Cited by 32 publications

References 36 publications

Complex-Valued Neural Networks With Nonparametric Activation Functions

Complex-Valued Neural Networks With Nonparametric Activation Functions

Is a Complex-Valued Stepsize Advantageous in Complex-Valued Gradient Learning Algorithms?

An Augmented Nonlinear LMS for Digital Self-Interference Cancellation in Full-Duplex Direct-Conversion Transceivers

Contact Info

Product

Resources

About