Autonomous Multiagent Space Exploration with High-Level Human Feedback

Abstract

Robotic space-exploration missions have always pushed the limits of science and technology, and will continue to do so by their very nature. Such missions are particularly challenging, as they operate in environments with high uncertainty, light-time delays, and high mission costs. Artificial-intelligence-based multiagent systems can alleviate these concerns by 1) creating autonomous multirobot teams that can function in uncertain environments, 2) navigating and operating without time-sensitive commands from Earth-bound scientists, and 3) spreading the mission cost across multiple platforms that will eliminate the danger of total mission loss in the case of a malfunctioning robot. In this work, a novel human in-the-loop cooperative coevolutionary algorithm is presented to train a multirobot system exploring an unknown environment. Autonomous robots learn to make low-level control decisions to maximize scientific data acquisition, whereas human scientists on Earth learn the changing mission profiles and provide highlevel objectives to the robots. Results demonstrate that the algorithm reduces the number of robots needed for a particular performance level tenfold compared to traditional cooperative coevolutionary algorithms for configurations of 10 or more rovers, resulting in significantly lower mission costs. Further, the trained multirobot system is extremely robust to noise, and 10% sensor and actuator noise (with and without sensor bias) has no statistically significant impact on system performance. Finally, the system is extremely robust to robot failures; for any percentage of robot failures between 10 and 90%, the percentage loss in performance is less than the percentage of failed robots.

It’s All about Reward: Contrasting Joint Rewards and Individual Reward in Centralized Learning Decentralized Execution Algorithms

“…The closest works addressing team rewards in cooperative settings that we could find include works on difference rewards which try to measure the impact of an individual agent's actions on the full system reward [12]. The high learnability, among other nice properties, makes difference rewards attractive but impractical, due to the required knowledge of the total system state [13][14][15].…”

Counterfactual Focused Learning

“…Multiagent teams have been effectively applied to many domains requiring coordination among team members such as robot soccer [1][2][3], the manipulation of large objects (such as boxes) [4][5][6], and joint exploration tasks [7][8][9][10].…”