Integrating unsupervised and reinforcement learning in human categorical perception: A computational model

Granato, Giovanni; Cartoni, Emilio; Rold, Federico Da; Mattera, Andrea; Baldassarre, Gianluca

doi:10.1371/journal.pone.0267838

Cited by 5 publications

(5 citation statements)

References 115 publications

(148 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unfortunately, a marked imbalance toward PEs in young adults remains even after a complete brain maturation. Consistent with the experimental evidence 31 , 72 , 73 , this could depend on a residual imbalanced effect of the sub-cortical structures on cortical systems.…”

Section: Discussionsupporting

confidence: 83%

“…Indeed, the three-component hypothesis focuses on representation manipulations rather than on motor variability. We have started to consider the representation of learning and sensory-motor differences in autistic population in 31 . At last, after each sorting attempt, a simulated ‘external operator’ component, knowing the correct sorting rule, receives the deck card and the chosen target card and returns positive or negative feedback to the model.…”

Section: Methodsmentioning

confidence: 99%

“…Note that this result agrees with theories and models on rule-based category learning 6 , 30 , suggesting that it is mostly based on a rule/representations switch process. In particular, rule-based category learning should not depend on specific categories/rules or shapes, provided that the participant has correctly acquired abstract and perceptual representations of them (we begin to investigate the representations learning issue in 31 ). Participants are required to sort each deck card, choosing one of the three categories.…”

Section: Methodsmentioning

confidence: 99%

See 2 more Smart Citations

A computational model of inner speech supporting flexible goal-directed behaviour in Autism

Granato

Borghi

Mattera

et al. 2022

Sci Rep

Self Cite

View full text Add to dashboard Cite

Experimental and computational studies propose that inner speech boosts categorisation skills and executive functions, making human behaviour more focused and flexible. In addition, many clinical studies highlight a relationship between poor inner-speech and an executive impairment in autism spectrum condition (ASC), but contrasting findings are reported. Here we directly investigate the latter issue through a previously implemented and validated computational model of the Wisconsin Cards Sorting Tests. In particular, the model was applied to explore potential individual differences in cognitive flexibility and inner speech contribution in autistic and neurotypical participants. Our model predicts that the use of inner-speech could increase along the life-span of neurotypical participants but would be reduced in autistic ones. Although we found more attentional failures (i.e., wrong behavioural rule switches) in autistic children/teenagers and more perseverative behaviours in autistic young/older adults, only autistic children and older adults exhibited a lower performance (i.e., fewer consecutive correct rule switches) than matched control groups. Overall, our results corroborate the idea that the reduced use of inner speech could represent a disadvantage for autistic children and autistic older adults. Moreover, the results suggest that cognitive-behavioural therapies should focus on developing inner speech skills in autistic children as this could provide cognitive support throughout their whole life span.

show abstract

Section: Discussionsupporting

confidence: 83%

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

A computational model of inner speech supporting flexible goal-directed behaviour in Autism

Granato

Borghi

Mattera

et al. 2022

Sci Rep

Self Cite

View full text Add to dashboard Cite

show abstract

“…The current paradigm should not show run-by-run learning ( Fig 7 ) because there was no strategy or information in this unskilled task to carry over to the following runs. However, future investigations could examine learning on a trial-by-trial basis [ 47 , 48 ]. Other approaches could adapt this paradigm to incorporate multi-dimensional strategies.…”

Section: Discussionmentioning

confidence: 99%

Simulated operant reflex conditioning environment reveals effects of feedback parameters

Kim,

Oblak,

Manella

et al. 2024

PLoS ONE

View full text Add to dashboard Cite

Operant conditioning of neural activation has been researched for decades in humans and animals. Many theories suggest two parallel learning processes, implicit and explicit. The degree to which feedback affects these processes individually remains to be fully understood and may contribute to a large percentage of non-learners. Our goal is to determine the explicit decision-making processes in response to feedback representing an operant conditioning environment. We developed a simulated operant conditioning environment based on a feedback model of spinal reflex excitability, one of the simplest forms of neural operant conditioning. We isolated the perception of the feedback signal from self-regulation of an explicit unskilled visuomotor task, enabling us to quantitatively examine feedback strategy. Our hypothesis was that feedback type, biological variability, and reward threshold affect operant conditioning performance and operant strategy. Healthy individuals (N = 41) were instructed to play a web application game using keyboard inputs to rotate a virtual knob representative of an operant strategy. The goal was to align the knob with a hidden target. Participants were asked to “down-condition” the amplitude of the virtual feedback signal, which was achieved by placing the knob as close as possible to the hidden target. We varied feedback type (knowledge of performance, knowledge of results), biological variability (low, high), and reward threshold (easy, moderate, difficult) in a factorial design. Parameters were extracted from real operant conditioning data. Our main outcomes were the feedback signal amplitude (performance) and the mean change in dial position (operant strategy). We observed that performance was modulated by variability, while operant strategy was modulated by feedback type. These results show complex relations between fundamental feedback parameters and provide the principles for optimizing neural operant conditioning for non-responders.

show abstract

“…The recurrent neural networks with the RL algorithm have been widely used to simulate behavior and neural activity of animals in cognitive tasks ( 57 , 58 ). In this framework, our model is trained with the RL in a way similar to that the animals learn the cognitive task with trial and error.…”

Section: Discussionmentioning

confidence: 99%

Category learning in a recurrent neural network with reinforcement learning

Zhang

Pan²,

Wang³

2022

Front. Psychiatry

View full text Add to dashboard Cite

It is known that humans and animals can learn and utilize category information quickly and efficiently to adapt to changing environments, and several brain areas are involved in learning and encoding category information. However, it is unclear that how the brain system learns and forms categorical representations from the view of neural circuits. In order to investigate this issue from the network level, we combine a recurrent neural network with reinforcement learning to construct a deep reinforcement learning model to demonstrate how the category is learned and represented in the network. The model consists of a policy network and a value network. The policy network is responsible for updating the policy to choose actions, while the value network is responsible for evaluating the action to predict rewards. The agent learns dynamically through the information interaction between the policy network and the value network. This model was trained to learn six stimulus-stimulus associative chains in a sequential paired-association task that was learned by the monkey. The simulated results demonstrated that our model was able to learn the stimulus-stimulus associative chains, and successfully reproduced the similar behavior of the monkey performing the same task. Two types of neurons were found in this model: one type primarily encoded identity information about individual stimuli; the other type mainly encoded category information of associated stimuli in one chain. The two types of activity-patterns were also observed in the primate prefrontal cortex after the monkey learned the same task. Furthermore, the ability of these two types of neurons to encode stimulus or category information was enhanced during this model was learning the task. Our results suggest that the neurons in the recurrent neural network have the ability to form categorical representations through deep reinforcement learning during learning stimulus-stimulus associations. It might provide a new approach for understanding neuronal mechanisms underlying how the prefrontal cortex learns and encodes category information.

show abstract

Integrating unsupervised and reinforcement learning in human categorical perception: A computational model

Cited by 5 publications

References 115 publications

A computational model of inner speech supporting flexible goal-directed behaviour in Autism

A computational model of inner speech supporting flexible goal-directed behaviour in Autism

Simulated operant reflex conditioning environment reveals effects of feedback parameters

Category learning in a recurrent neural network with reinforcement learning

Contact Info

Product

Resources

About