Shared environment representation for a human‐robot team performing information fusion

Abstract

This paper addresses the problem of building a shared environment representation by a human-robot team. Rich environment models are required in real applications for both autonomous operation of robots and to support human decision-making. Two probabilistic models are used to describe outdoor environment features such as trees: geometric ͑position in the world͒ and visual. The visual representation is used to improve data association and to classify features. Both models are able to incorporate observations from robotic platforms and human operators. Physically, humans and robots form a heterogeneous sensor network. In our experiments, the human-robot team consists of an unmanned air vehicle, a ground vehicle, and two human operators. They are deployed for an information gathering task and perform information fusion cooperatively. All aspects of the system including the fusion algorithms are fully decentralized. Experimental results are presented in form of the acquired multi-attribute feature map, information exchange patterns demonstrating human-robot information fusion, and quantitative model evaluation. Learned lessons from deploying the system in the field are also presented.

Categorical soft data fusion via variational Bayesian importance sampling with applications to cooperative search

“…This leads to fast and reliable estimates in both (μ,Σ) for (4), since IS can estimate any statistical moment of (3) if enough samples are drawn from an 'importance distribution' q(X, D = j) that is 'close' to (3). As the example below shows, IS is unreliable if q is chosen poorly, e.g.…”

“…Motivated by the studies of [2], [3], [6], this section describes the application of the VBIS fusion method to a cooperative mission involving a real human-robot team searching for five hidden stationary targets. The goal is for a single human agent to assist a single autonomous mobile robot in correctly locating and identifying all targets as quickly as possible.…”

“…While control and sensing capabilities for autonomous unmanned vehicles are always improving for such applications, human agents still serve key roles both as supervisory operators and as information-sharing agents [3]. Although human information provision is often geared towards classifying objects and abstract behaviors, less appreciated is the fact that humans can provide 'soft' information about continuous quantities of interest.…”

“…Ref. [3] developed a Bayesian method for fusing a human's continuous range/bearing estimates to tracked objects by modeling the 'human sensor' via linear regression models. Ref.…”

“…[3], [6], [7] explored incorporation of human information sources in the context of surveillance and search-andrescue applications. However, few formal methods have been proposed for 'low-level' human information fusion in the Bayesian framework, which has become increasingly popular for such applications.…”