Scalable Bayesian human-robot cooperation in mobile sensor networks

Bourgault, F.; Chokshi, A.; Wang, J.; Shah, Danelle C.; Schoenberg, Jonathan R.; Iyer, Rashi; Cedano, F.; Campbell, Mark

doi:10.1109/iros.2008.4651138

Cited by 22 publications

(18 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The pilot only needs to perform some tasks at certain times and be part of the overall human-machine control system. In [25], a collaborative work between human and robot systems for information gathering is presented. This problem also appears in other works such as [26], where human operators are integrated into a sensor network formed by a heterogeneous team of unmanned air and ground vehicles.…”

Section: Human-in-the-loop Controlmentioning

confidence: 99%

Human-in-the-Loop Model Predictive Control of an Irrigation Canal [Applications of Control]

2015

IEEE Control Syst.

View full text Add to dashboard Cite

U ntil now, advanced model-based control techniques have been predominantly employed to control problems that are relatively straightforward to model. Many systems with complex dynamics or containing sophisticated sensing and actuation elements can be controlled if the corresponding mathematical models are available, even if there is uncertainty in this information. Consequently, the application of model-based control strategies has flourished in numerous areas, including indus-The difficulty arises when there are components in the controlled system that are not easily modeled in a standard setting. One important and broad class of such systems is when humans are involved in the actuation process, in measurements, or in system dynamics. How to integrate the important role played by human operators in the control problem formulation is still an open question. Control methods typically rely on a fully automatic control operation that, in the case of humans being involved in the control system, is unrealistic. Likewise, fully manual control methods might compromise the performance of the system because of factors such as limitations on the availability of the operators to implement control actions or the lack of coordination when several operators are involved. Therefore, there is an incentive in terms of performance to provide a link between stateof-the-art control methods and human operation.

show abstract

Section: Human-in-the-loop Controlmentioning

confidence: 99%

Human-in-the-Loop Model Predictive Control of an Irrigation Canal [Applications of Control]

2015

IEEE Control Syst.

View full text Add to dashboard Cite

show abstract

“…Our approach uses Bayesian data fusion to exploit soft data with minimal effort on the part of human sensors and autonomous robotic sensor platforms. Such 'plug and play' human sensing for robot state estimation was explored in [1], [2] for restricted types of human observations, and has received increased attention in recent years [3], [4]. In this paper, we combine our recent work on Bayesian semantic natural language human data fusion [5], [6] with concepts from optimal active sensing, in order to develop new methods for interactive human-robot semantic sensing.…”

Section: Introductionmentioning

confidence: 99%

Closed-Loop Bayesian Semantic Data Fusion for Collaborative Human-Autonomy Target Search

Burks

Loefgren

Barbier

et al. 2018

2018 21st International Conference on Information Fusion (FUSION)

View full text Add to dashboard Cite

In search applications, autonomous unmanned vehicles must be able to efficiently reacquire and localize mobile targets that can remain out of view for long periods of time in large spaces. As such, all available information sources must be actively leveraged -including imprecise but readily available semantic observations provided by humans. To achieve this, this work develops and validates a novel collaborative human-machine sensing solution for dynamic target search. Our approach uses continuous partially observable Markov decision process (CPOMDP) planning to generate vehicle trajectories that optimally exploit imperfect detection data from onboard sensors, as well as semantic natural language observations that can be specifically requested from human sensors. The key innovation is a scalable hierarchical Gaussian mixture model formulation for efficiently solving CPOMDPs with semantic observations in continuous dynamic state spaces. The approach is demonstrated and validated with a real human-robot team engaged in dynamic indoor target search and capture scenarios on a custom testbed.

show abstract

“…Several modeling approaches have been developed to exploit soft human sensing across a variety of interfaces, e.g. verbally reported range and bearing for target localization [5]; verbal and sketch-based detection/no detection reports for target search [3,1]; and semantic language inputs for target localization [2]. While these works have largely focused on developing human sensor models and suitable data fusion algorithms for blending hard and soft data, relatively little work has been done on active soft sensing, i.e., intelligent querying of human sensors to gather information that would be most beneficial for complex machine planning and/or perception tasks.…”

Section: Introductionmentioning

confidence: 99%

Deep Value of Information Estimators for Collaborative Human-Machine Information Gathering

Lore

Sweet

Kumar

et al. 2016

2016 ACM/IEEE 7th International Conference on Cyber-Physical Systems (ICCPS)

View full text Add to dashboard Cite

Effective human-machine collaboration can significantly improve many learning and planning strategies for information gathering via fusion of 'hard' and 'soft' data originating from machine and human sensors, respectively. However, gathering the most informative data from human sensors without task overloading remains a critical technical challenge. In this context, Value of Information (VOI) is a crucial decision-theoretic metric for scheduling interaction with human sensors. We present a new Deep Learning based VOI estimation framework that can be used to schedule collaborative human-machine sensing with computationally efficient online inference and minimal policy hand-tuning. Supervised learning is used to train deep convolutional neural networks (CNNs) to extract hierarchical features from 'images' of belief spaces obtained via data fusion. These features can be associated with soft data query choices to reliably compute VOI for human interaction. The CNN framework is described in detail, and a performance comparison to a feature-based POMDP scheduling policy is provided. The practical feasibility of our method is also demonstrated on a mobile robotic search problem with language-based semantic human sensor inputs.

show abstract

Scalable Bayesian human-robot cooperation in mobile sensor networks

Cited by 22 publications

References 13 publications

Human-in-the-Loop Model Predictive Control of an Irrigation Canal [Applications of Control]

Human-in-the-Loop Model Predictive Control of an Irrigation Canal [Applications of Control]

Closed-Loop Bayesian Semantic Data Fusion for Collaborative Human-Autonomy Target Search

Deep Value of Information Estimators for Collaborative Human-Machine Information Gathering

Contact Info

Product

Resources

About