Restricted Minimum Error Entropy Criterion for Robust Classification

Li, Yuanhao; Chen, Badong; Yoshimura, Natsue; Koike, Yasuharu

doi:10.1109/tnnls.2021.3082571

Cited by 12 publications

(6 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this study, the fundamental inner product to measure the similarity is used, which is generalized from its vectors' application [33]. The inner product similarity between continuous pdfs f X (x) and g X (x) can be expressed as…”

Section: Restricted Meecmentioning

confidence: 99%

“…In fact, QMEE converges the prediction errors c j M j=1 to obtain a compact error distribution. Based on the method in [33], a predetermined codebook C = (0, −1, 1) implements QMEE to restrict errors to three positions and avoid the undesirable double-peak learning consequence. Therefore, the restricted MEE (RMEE) algorithm can be formulated as…”

Section: Restricted Meecmentioning

confidence: 99%

“…The detailed algorithm of the HQ-based optimization and its convergence analysis for RMEE are presented in [33].…”

Section: Restricted Meecmentioning

confidence: 99%

See 2 more Smart Citations

Robust Spike-Based Continual Meta-Learning Improved by Restricted Minimum Error Entropy Criterion

Yang

Tan

Chen

2022

Entropy

Self Cite

137

View full text Add to dashboard Cite

The spiking neural network (SNN) is regarded as a promising candidate to deal with the great challenges presented by current machine learning techniques, including the high energy consumption induced by deep neural networks. However, there is still a great gap between SNNs and the online meta-learning performance of artificial neural networks. Importantly, existing spike-based online meta-learning models do not target the robust learning based on spatio-temporal dynamics and superior machine learning theory. In this invited article, we propose a novel spike-based framework with minimum error entropy, called MeMEE, using the entropy theory to establish the gradient-based online meta-learning scheme in a recurrent SNN architecture. We examine the performance based on various types of tasks, including autonomous navigation and the working memory test. The experimental results show that the proposed MeMEE model can effectively improve the accuracy and the robustness of the spike-based meta-learning performance. More importantly, the proposed MeMEE model emphasizes the application of the modern information theoretic learning approach on the state-of-the-art spike-based learning algorithms. Therefore, in this invited paper, we provide new perspectives for further integration of advanced information theory in machine learning to improve the learning performance of SNNs, which could be of great merit to applied developments with spike-based neuromorphic systems.

show abstract

Section: Restricted Meecmentioning

confidence: 99%

Section: Restricted Meecmentioning

confidence: 99%

See 1 more Smart Citation

Robust Spike-Based Continual Meta-Learning Improved by Restricted Minimum Error Entropy Criterion

Yang

Tan

Chen

2022

Entropy

Self Cite

137

View full text Add to dashboard Cite

show abstract

“…The spiking patterns generated by biological neurons help us better understand how cortical neurons work. With the development of artificial general intelligence (Li et al, 2021 ; Yang et al, 2022a , b ) and its hardware (Yang et al, 2020 , 2021 ), the simulation of the structure and function of neurons in the brain has attracted great attention in various fields. Especially the study on the fundamental circuits of neurons, a lot of studies on the RC circuit have focused on the filter (Caves et al, 1977 ; Chen et al, 2021 ), the oscillator (Srinivasulu, 2012 ; Tsuruoka et al, 2020 ), the cell membrane model (Brosseau and Sabri, 2021 ), the RC network (Satish et al, 2020 ), the circuit structures (Hope et al, 2020 ; Mao et al, 2021 ), and the converter (Adapa et al, 2020 ; Liu et al, 2021b ).…”

Section: Introductionmentioning

confidence: 99%

Transient Response and Firing Behaviors of Memristive Neuron Circuit

et al. 2022

View full text Add to dashboard Cite

The signal transmission mechanism of the Resistor-Capacitor (RC) circuit is similar to the intracellular and extracellular signal propagating mechanism of the neuron. Thus, the RC circuit can be utilized as the circuit model of the neuron cell membrane. However, resistors are electronic components with the fixed-resistance and have no memory properties. A memristor is a promising neuro-morphological electronic device with nonvolatile, switching, and nonlinear characteristics. First of all, we consider replacing the resistor in the RC neuron circuit with a memristor, which is named the Memristor-Capacitor (MC) circuit, then the MC neuron model is constructed. We compare the charging and discharging processes between the RC and MC neuron circuits. Secondly, two models are compared under the different external stimuli. Finally, the synchronous and asynchronous activities of the RC and MC neuron circuits are performed. Extensive experimental results suggest that the charging and discharging speed of the MC neuron circuit is faster than that of the RC neuron circuit. Given sufficient time and proper external stimuli, the RC and MC neuron circuits can produce the action potentials. The synchronous and asynchronous phenomena in the two neuron circuits reproduce nonlinear dynamic behaviors of the biological neurons.

show abstract

“…The minimum error entropy (MEE) criterion 33 is an important learning criterion in information theoretic learning, which has been successfully applied in robust regression, classification, and adaptive filtering 34–36 . Chen et al 37 successfully develop a new Kalman‐type filter, called MEE‐KF, by using the MEE as the optimality criterion, which verified the excellent performance and robustness.…”

Section: Introductionmentioning

confidence: 99%

The research on channel estimation and signal‐noise ratio estimation based on minimum error entropy kalman filter for single carrier frequency domain equalization system

Pengyu

Xiang

Liang

et al. 2022

Int J Communication

View full text Add to dashboard Cite

In this paper, the channel estimation and signal-to-noise ratio (SNR) estimation technique of single-carrier frequency domain equalization (SC-FDE) system under low SNR in aeronautical multipath channel are studied, a SNR estimation algorithm which is easy to implement in engineering and an improved LS channel estimation algorithm based on Kalman filter using minimum error entropy (MEE-KF) are proposed. This paper first introduces the SC-FDE system and introduces the principle of MEE-KF, and then, the channel estimation flow based on MEE-KF is obtained by combining it with the traditional LS channel estimation algorithm, which makes the estimation results perform better. Simulation results show that after getting more accurate noise variance, the channel estimation results can better follow the changes of the channel after MEE-KF processing, so as to resist the doppler frequency offset effect and make the channel estimation results more accurate, that is the channel response results of the data part can be closer to the real situation, so that the communication performance of SC-FDE system has also been greatly improved.

show abstract

Restricted Minimum Error Entropy Criterion for Robust Classification

Cited by 12 publications

References 47 publications

Robust Spike-Based Continual Meta-Learning Improved by Restricted Minimum Error Entropy Criterion

Robust Spike-Based Continual Meta-Learning Improved by Restricted Minimum Error Entropy Criterion

Transient Response and Firing Behaviors of Memristive Neuron Circuit

The research on channel estimation and signal‐noise ratio estimation based on minimum error entropy kalman filter for single carrier frequency domain equalization system

Contact Info

Product

Resources

About