Confidence-controlled Hebbian learning efficiently extracts category membership from stimuli encoded in view of a categorization task

Abstract

In experiments on perceptual decision-making, individuals learn a categorization task through trial-and-error protocols. We explore the capacity of a decision-making attractor network to learn a categorization task through reward-based, Hebbian type, modifications of the weights incoming from the stimulus encoding layer. For the latter, we assume a standard layer of a large number of stimulus specific neurons. Within the general framework of Hebbian learning, authors have hypothesized that the learning rate is modulated by the reward at each trial. Surprisingly, we find that, when the coding layer has been optimized in view of the categorization task, such reward-modulated Hebbian learning (RMHL) fails to extract efficiently the category membership. In a previous work we showed that the attractor neural networks nonlinear dynamics accounts for behavioral confidence in sequences of decision trials. Taking advantage of these findings, we propose that learning is controlled by confidence, as computed from the neural activity of the decision-making attractor network. Here we show that this confidence-controlled, reward-based, Hebbian learning efficiently extracts categorical information from the optimized coding layer. The proposed learning rule is local, and, in contrast to RMHL, does not require to store the average rewards obtained on previous trials. In addition, we find that the confidence-controlled learning rule achieves near optimal performance.

Surprise: a unified theory and experimental predictions

“…Importantly, our theoretical analysis provides foundations for biological mechanism of surprise computation and surprise-modulated learning (Berlemont & Nadal, 2021;Frémaux & Gerstner, 2016;Iigaya, 2016;Soltani & Izquierdo, 2019). For example, it has been argued that the computation of observation-mismatch surprise measures is biologically more plausible than more abstract measures such as Shannon surprise (Iigaya, 2016).…”

“…(see Fiser et al, 2010;Knill and Pouget, 2004;Soltani and Wang, 2010 for examples of neural models of probabilistic inference), 'how can surprisemodulated adaptive learning be implemented at the level of synaptic plasticity?' (Berlemont & Nadal, 2021;Gerstner et al, 2018;Iigaya, 2016;Illing et al, 2021), and 'how can surprise-seeking exploration strategies be implemented in the brain?' (see Basanisi et al, 2020 for an example).…”

Categorical Perception: A Groundwork for Deep Learning

“…Depending on the constraints specific to the system under consideration, minimization of the cost leads to a neural code such that F code (x) is some increasing function of F cat (x). For some constraints one gets F code (x) ∝ F cat (x) as optimum, but other constraints may lead to other relationships -see Bonnasse-Gahot and Nadal (2008; Berlemont and Nadal (2020). Efficient coding with respect to optimal classification is thus obtained by essentially matching the two metrics.…”