Coordinating Teams in Uncertain Environments: A Hybrid BDI-POMDP Approach

Abstract

Abstract. Distributed partially observable Markov decision problems (POMDPs) have emerged as a popular decision-theoretic approach for planning for multiagent teams, where it is imperative for the agents to be able to reason about the rewards (and costs) for their actions in the presence of uncertainty. However, finding the optimal distributed POMDP policy is computationally intractable (NEXPComplete). This paper is focussed on a principled way to combine the two dominant paradigms for building multiagent team plans, namely the "belief-desireintention" (BDI) approach and distributed POMDPs. In this hybrid BDI-POMDP approach, BDI team plans are exploited to improve distributed POMDP tractability and distributed POMDP-based analysis improves BDI team plan performance. Concretely, we focus on role allocation, a fundamental problem in BDI teams -which agents to allocate to the different roles in the team. The hybrid BDI-POMDP approach provides three key contributions. First, unlike prior work in multiagent role allocation, we describe a role allocation technique that takes into account future uncertainties in the domain. The second contribution is a novel decomposition technique, which exploits the structure in the BDI team plans to significantly prune the search space of combinatorially many role allocations. Our third key contribution is a significantly faster policy evaluation algorithm suited for our BDI-POMDP hybrid approach. Finally, we also present experimental results from two domains: mission rehearsal simulation and RoboCupRescue disaster rescue simulation. In the RoboCupRescue domain, we show that the role allocation technique presented in this paper is capable of performing at human expert levels by comparing with the allocations chosen by humans in the actual RoboCupRescue simulation environment.

Recognizing and learning models of social exchange strategies for the regulation of social interactions in open agent societies

“…4 Following, we internalized the regulation mechanism in the agents, through a BDI-POMDP hybrid model [38], going toward the self-regulation of social exchanges. A comparison between social exchange regulation models was also presented by Pereira et al [40] Observe that hybrid models have been proposed in the literature in order to take advantages of both POMDPs and BDI architecture (see, e.g., [33,34,37,[51][52][53][54][55]). 5 Simari and Parsons [53] showed that the plans derived from an optimal policy are exactly the same adopted by a BDI agent that selects plans with the highest utility, and that operate with an optimal reconsideration strategy.…”

“…For that, we have built on hybrid BDI-POMDP agent models, defined over the BDI (Beliefs, Desires, Intentions) architecture [5,43,56] with plans derived from POMDP (Partially Observable Markov Decision Processes) [30] models of social exchange strategies, in the line of the works of Simari and Parsons [53] and other authors [33,34,37,51,52,54,55].…”