A Novel Concept for the Study of Heterogeneous Robotic Swarms warm robotics systems are characterized by decentralized control, limited communication between robots, use of local information, and emergence of global behavior. Such systems have shown their potential for flexibility and robustness [1]-[3]. However, existing swarm robotics systems are by and large still limited to displaying simple proof-of-concept behaviors under laboratory conditions. It is our contention that one of the factors holding back swarm robotics research is the almost universal insistence on homogeneous system components. We believe that swarm robotics designers must embrace heterogeneity if they ever want swarm robotics systems to approach the complexity required of real-world systems. To date, swarm robotics systems have almost exclusively comprised physically and behaviorally undifferentiated agents. This design decision has its roots in ethological models of self-organizing natural systems. These models serve as inspiration for swarm robotics system designers, but are often highly abstract simplifications of natural systems and, to date, have largely assumed homogeneous agents. Selected dynamics of the systems under study are shown to emerge from the interactions of identical system components, ignoring the heterogeneities (physical, spatial, functional, and informational) that one can find in almost any natural system. The field of swarm robotics currently lacks methods and tools with which to study and leverage the heterogeneity that is present in natural systems. To remedy this deficiency, we propose swarmanoid, an innovative swarm robotics system composed of three different robot types with complementary skills: foot-bots are small autonomous robots specialized in moving on both even and uneven terrains, capable of self-assembling and of transporting objects or other robots; hand-bots are autonomous robots capable of climbing some vertical surfaces and manipulating small objects; and eye-bots are autonomous flying robots that can attach to an indoor ceiling, capable of analyzing the environment from a privileged position to S

Reinforcement learning for partially observable Markov decision problems (POMDPs) is a challenge as it requires policies with an internal state. Traditional approaches suffer significantly from this shortcoming and usually make strong assumptions on the problem domain such as perfect system models, state-estimators and a Markovian hidden system. Recurrent neural networks (RNNs) offer a natural framework for dealing with policy learning using hidden state and require only few limiting assumptions. As they can be trained well using gradient descent, they are suited for policy gradient approaches.In this paper, we present a policy gradient method, the Recurrent Policy Gradient which constitutes a model-free reinforcement learning method. It is aimed at training limited-memory stochastic policies on problems which require long-term memories of past observations. The approach involves approximating a policy gradient for a recurrent neural network by backpropagating return-weighted characteristic eligibilities through time. Using a ''Long ShortTerm Memory'' RNN architecture, we are able to outperform previous RL methods on three important benchmark tasks. Furthermore, we show that using history-dependent baselines helps reducing estimation variance significantly, thus enabling our approach to tackle more challenging, highly stochastic environments.

Most previous work on artificial curiosity (AC) and intrinsic motivation focuses on basic concepts and theory. Experimental results are generally limited to toy scenarios, such as navigation in a simulated maze, or control of a simple mechanical system with one or two degrees of freedom. To study AC in a more realistic setting, we embody a curious agent in the complex iCub humanoid robot. Our novel reinforcement learning (RL) framework consists of a state-of-the-art, low-level, reactive control layer, which controls the iCub while respecting constraints, and a high-level curious agent, which explores the iCub's state-action space through information gain maximization, learning a world model from experience, controlling the actual iCub hardware in real-time. To the best of our knowledge, this is the first ever embodied, curious agent for real-time motion planning on a humanoid. We demonstrate that it can learn compact Markov models to represent large regions of the iCub's configuration space, and that the iCub explores intelligently, showing interest in its physical constraints as well as in objects it finds in its environment.

No abstract

Abstract-We present a cooperative navigation algorithm for robotic swarms. Its purpose is to let a robot find a given target robot, while being guided by the other robots of the swarm. The system is based on wireless communication: the robots forward messages containing navigation information over the ad hoc network among them, and the searching robot uses this information to find its target. We study the algorithm in two different scenarios. In the first scenario, a single searching robot needs to find a single target, while all other robots are involved in tasks of their own. We show that the communication based navigation system allows the robots of the swarm to guide the searching robot without the need to adapt their own movements. In the second scenario, we study collective navigation: all robots of the swarm need to navigate back and forth between two targets. We show that in this case, the proposed navigation algorithm gives rise to synergies in robot navigation, and lets the swarm self-organize into a robust dynamic structure. This selforganization improves navigation efficiency, and lets the swarm find shortest paths in cluttered environments. We test our system both in simulation and on real robots.

No abstract

We present curiosity-driven, autonomous acquisition of tactile exploratory skills on a biomimetic robot finger equipped with an array of microelectromechanical touch sensors. Instead of building tailored algorithms for solving a specific tactile task, we employ a more general curiosity-driven reinforcement learning approach that autonomously learns a set of motor skills in absence of an explicit teacher signal. In this approach, the acquisition of skills is driven by the information content of the sensory input signals relative to a learner that aims at representing sensory inputs using fewer and fewer computational resources. We show that, from initially random exploration of its environment, the robotic system autonomously develops a small set of basic motor skills that lead to different kinds of tactile input. Next, the system learns how to exploit the learned motor skills to solve supervised texture classification tasks. Our approach demonstrates the feasibility of autonomous acquisition of tactile skills on physical robotic platforms through curiosity-driven reinforcement learning, overcomes typical difficulties of engineered solutions for active tactile exploration and underactuated control, and provides a basis for studying developmental learning through intrinsic motivation in robots.

Background/Aims: To evaluate the use of the NeuroMate stereotactic robot with a novel ultrasound registration system for movement disorder surgery (MDS). Methods: Using the robot in a frameless mode, 51 patients underwent MDS. Surgical planning was carried out using MRI data obtained more than 24 h before surgery. Results: 37 out of 50 targets in the subthalamic nucleus were satisfactorily identified with a single microelectrode trajectory and the final electrode positions were at a mean distance of 1.7 mm from the calculated target. There was a significant improvement in motor scores of the Unified Parkinson’s Disease Rating Scale III (off medication) at 6 (43%) and 18 months (51.7%) compared to pre-operative scores (p < 0.05). Conclusions: The frameless robot using only MRI data can be used for MDS. The temporal separation of imaging from the surgical procedure provides additional time for detailed image analysis and planning.