Robots learn as they interact with humans. Consider a human teleoperating an assistive robot arm: as the human guides and corrects the arm's motion, the robot gathers information about the human's desired task. But how does the human know what their robot has inferred? Today's approaches often focus on conveying intent: for instance, using legible motions or gestures to indicate what the robot is planning. However, closing the loop on robot inference requires more than just revealing the robot's current policy: the robot should also display the alternatives it thinks are likely, and prompt the human teacher when additional guidance is necessary. In this paper we propose a multimodal approach for communicating robot inference that combines both passive and active feedback. Specifically, we leverage information-rich augmented reality to passively visualize what the robot has inferred, and attention-grabbing haptic wristbands to actively prompt and direct the human's teaching. We apply our system to shared autonomy tasks where the robot must infer the human's goal in real-time. Within this context, we integrate passive and active modalities into a single algorithmic framework that determines when and which type of feedback to provide. Combining both passive and active feedback experimentally outperforms single modality baselines; during an in-person user study, we demonstrate that our integrated approach increases how efficiently humans teach the robot while simultaneously decreasing the amount of time humans spend interacting with the robot. Videos here: https://youtu.be/swq_u4iIP-g
Echolocating bat species that inhabit dense vegetation are promising model systems for achieving autonomy in complex natural environments. To replicate the skills of the bats in extracting information about complex environment that can support autonomous navigation in a man-made system three aspects deserve particular attention: (i) encoding of relevant sensory information; (ii) information extraction; and (iii) integration of information encoding and extraction with each other and the respective context. To facilitate progress towards understanding these issues, a biomimetic sonar system is being developed that is aimed at recreating the flexibility and variability that bats exhibit in the configuration of the emission and reception baffles of their biosonar systems (i.e., noseleaves and pinnae). Analyzing the significance of these dynamic effects on sensory information encoding requires an understanding of the stimulus ensemble of biosonar tasks in natural environment. To this end, the biomimetic sonar system has been used to collect large data sets with echoes that pertain to fundamental navigation tasks such as localization and passageway finding. Deep-learning methods have been shown to be a good match for analyzing this echo data. Finally, ongoing research is directed at data acquisition and inference with the specifics of a given habitat and biosonar task.
A sensory map of the world is a key component of any cognitive system. For cognitive systems navigating the physical world, this map is typically assumed to be a representation of the three-dimensional geometries in the environment. For vision-based maps, this is not an overly difficult goal to accomplish, since images have already two dimensions and hence only depth has to be inferred from additional clues. For bats using biosonar to navigate complex natural environments, such as dense vegetation, reconstruction of scatterer geometry is likely an ill-posed problem under the constraints of the biosonar systems (e.g., on beamwidth and spatial sampling). In these cases, biosonar echoes are “clutter,” i.e., signals that must be regarded as unpredictable due to lack of knowledge. Deep learning offers an opportunity to explore the information content of these echoes and hence understand the sensory map of bat biosonar on small and large scales. In addition, the biosonar systems of many bat species that live in dense vegetation are highly active senses, where time-variant signal transformations of the emitted pulses and the returning echoes are controlled in a perception-action loop. Again, deep learning can be used to understand the information that is encoded in this loop.
It has been observed that bat species such as the horseshoe bats (Rhinolophidae) that are capable of biosonar-based navigation in dense vegetation deform their noseleaves and pinnae during echolocation. To investigate the impact of these deformations on the encoding of sensory information in the biosonar echoes, a biomimetic robot has been developed to replicate this unique peripheral dynamics. Horseshoe bats have about twenty muscles on each pinna and demonstrate a considerable amount of variability in the motions of these structures. To replicate even a small portion of this flexibility and variability, an actuation mechanism with a small footprint on the deforming structure is necessary. In a prior version of the biomimetic robot, soft-robotic pneumatic actuators were used, but the size of these devices limited their number per pinna and relative orientation. To alleviate these issues, a tension-controlled actuation concept has been developed that consists a bank of servo motors that are connected to the pinna via tendons. At present, this system is designed for five degrees of freedom per pinna and three for the noseleaf. Ongoing work seeks to address how the tension-based actuation system can be controlled to recreate the variability in the noseleaf and pinna deformations of bats.
Robots learn as they interact with humans. Consider a human teleoperating an assistive robot arm: as the human guides and corrects the arm's motion, the robot gathers information about the human's desired task. But how does the human know what their robot has inferred? Today's approaches often focus on conveying intent: for instance, using legible motions or gestures to indicate what the robot is planning. However, closing the loop on robot inference requires more than just revealing the robot's current policy: the robot should also display the alternatives it thinks are likely, and prompt the human teacher when additional guidance is necessary. In this paper we propose a multimodal approach for communicating robot inference that combines both passive and active feedback. Specifically, we leverage information-rich augmented reality to passively visualize what the robot has inferred, and attention-grabbing haptic wristbands to actively prompt and direct the human's teaching. We apply our system to shared autonomy tasks where the robot must infer the human's goal in real-time. Within this context, we integrate passive and active modalities into a single algorithmic framework that determines when and which type of feedback to provide. Combining both passive and active feedback experimentally outperforms single modality baselines; during an in-person user study, we demonstrate that our integrated approach increases how efficiently humans teach the robot while simultaneously decreasing the amount of time humans spend interacting with the robot. Videos here: https://youtu.be/swq_u4iIP-g
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.