Reinforcement learning (RL) has become a popular paradigm for modeling animal behavior, analyzing neuronal representations, and studying their emergence during learning. This development has been fueled by advances in understanding the role of RL in both the brain and artificial intelligence. However, while in machine learning a set of tools and standardized benchmarks facilitate the development of new methods and their comparison to existing ones, in neuroscience, the software infrastructure is much more fragmented. Even if sharing theoretical principles, computational studies rarely share software frameworks, thereby impeding the integration or comparison of different results. Machine learning tools are also difficult to port to computational neuroscience since the experimental requirements are usually not well aligned. To address these challenges we introduce CoBeL-RL, a closed-loop simulator of complex behavior and learning based on RL and deep neural networks. It provides a neuroscience-oriented framework for efficiently setting up and running simulations. CoBeL-RL offers a set of virtual environments, e.g., T-maze and Morris water maze, which can be simulated at different levels of abstraction, e.g., a simple gridworld or a 3D environment with complex visual stimuli, and set up using intuitive GUI tools. A range of RL algorithms, e.g., Dyna-Q and deep Q-network algorithms, is provided and can be easily extended. CoBeL-RL provides tools for monitoring and analyzing behavior and unit activity, and allows for fine-grained control of the simulation via interfaces to relevant points in its closed-loop. In summary, CoBeL-RL fills an important gap in the software toolbox of computational neuroscience.

No abstract

Reinforcement learning (RL) has become a popular paradigm for modeling animal behavior, analyzing neuronal representations, and studying their emergence during learning. This development has been fueled by advances in understanding the role of RL in both the brain and artificial intelligence. However, while in machine learning a set of tools and standardized benchmarks facilitate the development of new methods and their comparison to existing ones, in neuroscience, the software infrastructure is much more fragmented. Even if sharing theoretical principles, computational studies rarely share software frameworks, thereby impeding the integration or comparison of different results. Machine learning tools are also difficult to port to computational neuroscience since the experimental requirements are usually not well aligned. To address these challenges we introduce CoBeL-RL, a closed-loop simulator of complex behavior and learning based on RL and deep neural networks. It provides a neuroscience-oriented framework for efficiently setting up and running simulations. CoBeL-RL offers a set of virtual environments, e.g. T-maze and Morris water maze, which can be simulated at different levels of abstraction, e.g. a simple gridworld or a 3D environment with complex visual stimuli, and set up using intuitive GUI tools. A range of RL algorithms, e.g. Dyna-Q and deep Q-network algorithms, is provided and can be easily extended. CoBeL-RL provides tools for monitoring and analyzing behavior and unit activity, and allows for fine-grained control of the simulation via interfaces to relevant points in its closed-loop. In summary, CoBeL-RL fills an important gap in the software toolbox of computational neuroscience.

No abstract

Simultaneous localization and mapping (SLAM) refers to techniques for autonomously constructing a map of an unknown environment while, at the same time, locating the robot in this map. RatSLAM, a prevalent method, is based on the navigation system found in rodent brains. It has served as a base algorithm for other bioinspired approaches, and its implementation has been extended to incorporate new features. This work proposes xRatSLAM: an extensible, parallel, open-source framework applicable for developing and testing new RatSLAM variations. Tests were carried out to evaluate and validate the proposed framework, allowing the comparison of xRatSLAM with OpenRatSLAM and assessing the impact of replacing framework components. The results provide evidence that the maps produced by xRatSLAM are similar to those produced by OpenRatSLAM when they are fed with the same input parameters, which is a positive result, and that implemented modules can be easily changed without impacting other parts of the framework.

No abstract

To successfully perform autonomous navigation, mobile agents must solve the Simultaneous Localization and Mapping (SLAM) problem. However, acquiring the map in a single SLAM session may not be possible, thus the map may be incrementally built over multiple sessions. Two solutions could be considered to solve the multisession SLAM problem: (i) the robot must localize itself in the previously stored map before the new session starts; (ii) it can start a new map and merge it with the map from the previous sessions. To date, only scenario (i) has been addressed by RatSLAM, an algorithm inspired by the navigation system in rodent brains. Therefore, this work proposes a multisession solution that solves both scenarios. A new mechanism merges the data from the RatSLAM structures of the current mapping session with those previously stored if there are connections between these paths. This approach was tested in four different scenarios, from virtual controlled environments to real-world environments with two, three, and five sessions. The robot started in an unfamiliar location for each mapping session, but it also works if the agent starts in a known place, scenario (ii) and (i), respectively. For all experiments, the entire map was consistently obtained. Furthermore, the proposed approach updates and enhances the previous session’s map in real-world environments. Therefore, the proposed approach may be a multiple SLAM session solution for the RatSLAM algorithm.

No abstract