Adaptive Stochastic Resonance in Noisy Neurons Based on Mutual Information

Abstract

Abstract-Noise can improve how memoryless neurons process signals and maximize their throughput information. Such favorable use of noise is the so-called "stochastic resonance" or SR effect at the level of threshold neurons and continuous neurons. This paper presents theoretical and simulation evidence that 1) lone noisy threshold and continuous neurons exhibit the SR effect in terms of the mutual information between random input and output sequences, 2) a new statistically robust learning law can find this entropy-optimal noise level, and 3) the adaptive SR effect is robust against highly impulsive noise with infinite variance. Histograms estimate the relevant probability density functions at each learning iteration. A theorem shows that almost all noise probability density functions produce some SR effect in threshold neurons even if the noise is impulsive and has infinite variance. The optimal noise level in threshold neurons also behaves nonlinearly as the input signal amplitude increases. Simulations further show that the SR effect persists for several sigmoidal neurons and for Gaussian radial-basis-function neurons.Index Terms-Alpha-stable noise, impulsive noise, infinite-variance statistics, mutual information, noise processing, sigmoidal neurons and radial basis functions, stochastic gradient learning, stochastic resonance (SR), threshold neurons. I. NOISE AND ADAPTIVE STOCHASTIC RESONANCE NOISE is an unwanted signal or source of energy. Scientists and engineers have largely tried to filter noise or cancel it or mask it out of existence. The Noise Pollution Clearinghouse condemns noise outright: "Noise is unwanted sound. It is derived from the Latin word 'nausea' meaning seasickness. Noise is among the most pervasive pollutants today. Noise from road traffic, jet planes, jet skis, garbage trucks, construction equipment, manufacturing processes, lawn mowers, leaf blowers, and boom boxes, to name a few, are among the unwanted sounds that are routinely broadcast into the air."The Fig. 1 shows how uniform pixel noise can improve our subjective perception of an image. The system quantizes the original gray-scale "Lena" image into a binary image of black and white pixels. It emits a white pixel as output if the input gray-scale pixel equals or exceeds a threshold. It emits a black pixel as output if the input gray-scale pixel falls below the threshold. This quantizer is biased because it does not set the threshold at the midpoint of the gray scale. So the quantized version of the original image contains almost no information. A small level of noise sharpens the image contours and helps fill in features when it adds to the original image before the system applies the threshold. Too much noise swamps the image and degrades its contours. Gammaitoni [29] and others [70] have proposed a dithering argument for this SR effect and still others [55] have applied this argument to still images. The argument involves adding dither noise to a signal before quantization. Consider gray-scale pixel and binary output pixel wi...

Optimal Noise Benefit in Composite Hypothesis Testing under Different Criteria

“…The following corollary shows the link between the optimization noise-enhanced problem in Equations (18) and (19) and that in (10) and (11).…”

“…The performance boost of a noise-enhanced system has also been observed in numerous signal detection problems; for example, when adjusting the background noise level or injecting additive noise to the input, the output of the system can be improved in some cases [10][11][12][13][14][15]. The improvements obtained via noise can be measured by various metrics, such as an increase in mutual information (MI) [16][17][18][19], output signal-to-noise ratio (SNR) [20][21][22], or detection probability [23][24][25][26][27][28][29], or a decrease in Bayes risk [30][31][32] or error probability [33].…”

Noise Enhanced Hypothesis-Testing in the Restricted Bayesian Framework

“…7 If there are multiple such n 's (n 's), then the one that minimizes F (n ) (F (n )) should be chosen.…”

Optimal Noise Enhanced Signal Detection in a Unified Framework

“…In the signal detection problem, researchers commonly care about how to increase the output signal-to-noise (SNR) [7][8][9][10][11], the mutual information [12,13], or detection probability with a constant false alarm rate [14][15][16][17][18][19][20], or how to decrease the Bayes risk [21,22] or the probability of error [23] by adding additive noise to the input of system or changing the background noise level. As presented in [8], the output SNR obtained by adding suitable noise to the input of system is higher than the input SNR.…”