Structural Estimation of Partially Observable Markov Decision Processes

Chang, Yanling; Garcia, Alfredo; Wang, Zhide; Sun, Lu

doi:10.1109/tac.2022.3217908

Cited by 1 publication

(1 citation statement)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the context of decision making, scholars have emphasized the importance of not only continuously detecting and analyzing the environment but also incorporating predictions and estimations of other agents' behaviors. To address these multifaceted task requirements, various models such as partially observable Markov decision processes (POMDPs) [21], interactive POMDPs [22], and interactive dynamic influence diagrams [23] have been proposed. These predictive models enable decision making based on anticipated environmental and agent behaviors.…”

Section: Literature Reviewmentioning

confidence: 99%

Enhancing Autonomous Underwater Vehicle Decision Making through Intelligent Task Planning and Behavior Tree Optimization

Yu,

Wang,

Cao

et al. 2024

JMSE

View full text Add to dashboard Cite

The expansion of underwater scenarios and missions highlights the crucial need for autonomous underwater vehicles (AUVs) to make informed decisions. Therefore, developing an efficient decision-making framework is vital to enhance productivity in executing complex tasks within tight time constraints. This paper delves into task planning and reconstruction within the AUV control decision system to enable intelligent completion of intricate underwater tasks. Behavior trees (BTs) offer a structured approach to organizing the switching structure of a hybrid dynamical system (HDS), originally introduced in the computer game programming community. In this research, an intelligent search algorithm, MCTS-QPSO (Monte Carlo tree search and quantum particle swarm optimization), is proposed to bolster the AUV’s capacity in planning complex task decision control systems. This algorithm tackles the issue of the time-consuming manual design of control systems by effectively integrating BTs. By assessing a predefined set of subtasks and actions in tandem with the complex task scenario, a reward function is formulated for MCTS to pinpoint the optimal subtree set. The QPSO algorithm is then leveraged for subtree integration, treating it as an optimal path search problem from the root node to the leaf node. This process optimizes the search subtree, thereby enhancing the robustness and security of the control architecture. To expedite search speed and algorithm convergence, this paper recommends reducing the search space by pre-grouping conditions and states within the behavior tree. The efficacy and superiority of the proposed algorithm are validated through security and timeliness evaluations of the BT, along with comparisons with other algorithms for automatic AUV decision control behavior tree design. Ultimately, the effectiveness and superiority of the proposed algorithm are corroborated through simulations on a multi-AUV complex task platform, showcasing its practical applicability and efficiency in real-world underwater scenarios.

show abstract

Section: Literature Reviewmentioning

confidence: 99%