CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning

Abstract

Reinforcement learning (RL) has become a popular paradigm for modeling animal behavior, analyzing neuronal representations, and studying their emergence during learning. This development has been fueled by advances in understanding the role of RL in both the brain and artificial intelligence. However, while in machine learning a set of tools and standardized benchmarks facilitate the development of new methods and their comparison to existing ones, in neuroscience, the software infrastructure is much more fragmented. Even if sharing theoretical principles, computational studies rarely share software frameworks, thereby impeding the integration or comparison of different results. Machine learning tools are also difficult to port to computational neuroscience since the experimental requirements are usually not well aligned. To address these challenges we introduce CoBeL-RL, a closed-loop simulator of complex behavior and learning based on RL and deep neural networks. It provides a neuroscience-oriented framework for efficiently setting up and running simulations. CoBeL-RL offers a set of virtual environments, e.g., T-maze and Morris water maze, which can be simulated at different levels of abstraction, e.g., a simple gridworld or a 3D environment with complex visual stimuli, and set up using intuitive GUI tools. A range of RL algorithms, e.g., Dyna-Q and deep Q-network algorithms, is provided and can be easily extended. CoBeL-RL provides tools for monitoring and analyzing behavior and unit activity, and allows for fine-grained control of the simulation via interfaces to relevant points in its closed-loop. In summary, CoBeL-RL fills an important gap in the software toolbox of computational neuroscience.

Interaction between neuroscience and happiness: assessment from Artificial Intelligence advances

“…In this sense, AI is enabling more accurate imaging examinations and the treatment of diseases affecting the nervous system. (33) Neuroscience in psychology allows understanding of how the brain behaves and influences emotions and behaviors; it also enables the development of psychological theories with a more biological foundation since behavior is currently being linked to brain activity in real-time thanks to tools such as functional magnetic resonance imaging (fMRI) and electroencephalography (EEG). The integration of these two areas of knowledge facilitates the understanding of the human mind, both theoretically and practically.…”

Competition and Integration of Visual and Goal Vector Signals for Spatial Navigation

“…To aid in localization, the walls of the enclosure were marked with distinct colors, which acted as distal landmarks. We used CoBeL-RL (Closed-loop simulator of complex behavior and learning based on reinforcement learning and deep neural networks) (Diekmann et al, 2023) to generate the simulation environments and train the agents. The agent received visual input and goal vector information from the simulation environment at each time step.…”

Global remapping emerges as the mechanism for renewal of context-dependent behavior in a reinforcement learning model

“…All simulations were based on the CoBeL-RL simulation framework (Diekmann et al, 2023). Visual feedback was provided to the artificial agent in the form of naturalistic images (Fig.…”