A survey of knowledge‐based sequential decision‐making under uncertainty

Zhang, Shiqi; Sridharan, Mohan

doi:10.1002/aaai.12053

Cited by 4 publications

(4 citation statements)

References 80 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The main task in decision‐making algorithms is that it must be fast enough to have real‐time. According to [33], decision‐making systems for self‐driving require four criteria, rapidity in the planned decision, coherency for avoiding unnecessary actions, and providentness which means that the module should foresee how the situation will evolve after some time/manoeuvres and include it in the decision‐ making, and finally, the predictability.…”

Section: Discussionmentioning

confidence: 99%

“…Bayesian Networks (BNs) are typically part of Directed Acyclic Graphs [30]. In general, Decision Networks combine BNs with additional node types for actions and utilities [31], whereas Dynamic Networks 'DNs' provide support for probabilistic reasoning and decision-making in systems with uncertainty, and offer the capability to integrate multiple decision criteria, making them highly effective for path planning tasks [32,33]. According to [34], BNs can be structured into two levels, the situation assessment level to infer the current situation state based on the risk assessment, and the decisionmaking strategy level to deduce the manoeuvring decisions.…”

Section: Probabilistic Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Advancing autonomous vehicle control systems: An in‐depth overview of decision‐making and manoeuvre execution state of the art

Abdallaoui,

Ikaouassen,

Kribèche

et al. 2023

The Journal of Engineering

View full text Add to dashboard Cite

This abstract discusses the significant progress made in autonomous vehicles, focusing on decision‐making systems and control algorithms. It explores recent advances, challenges, and contributions in the field, emphasizing the need for precise navigation and control. The paper covers various methodologies, including rule‐based methods, machine learning, deep learning, probabilistic approaches, and hybrid approaches, examining their applications and effectiveness in ensuring safe navigation. Additionally, it reviews ongoing research efforts, emerging trends, and persistent challenges related to decision‐making and manoeuvre execution in autonomous vehicles, addressing complex topics such as sensor measurement uncertainty, dynamic environment modelling, real‐time responsiveness, and safe interactions with other road users. The objective is to provide a comprehensive overview of the state of the art in autonomous vehicle navigation and control for readers.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Probabilistic Methodsmentioning

confidence: 99%

Advancing autonomous vehicle control systems: An in‐depth overview of decision‐making and manoeuvre execution state of the art

Abdallaoui,

Ikaouassen,

Kribèche

et al. 2023

The Journal of Engineering

View full text Add to dashboard Cite

show abstract

“…Pateria et al (2022) survey hierarchical methods that come under modular decompositions in our framework. Zhang and Sridharan (2022) survey methods that leverage reasoning and declarative knowledge for sequential decision-making, including RL. A class of such methods falls under the relational decompositions in our framework.…”

Section: Related Workmentioning

confidence: 99%

Structure in Deep Reinforcement Learning: A Survey and Open Problems

Mohan,

Zhang,

Lindauer

2024

jair

View full text Add to dashboard Cite

Reinforcement Learning (RL), bolstered by the expressive capabilities of Deep Neural Networks (DNNs) for function approximation, has demonstrated considerable success in numerous applications. However, its practicality in addressing various real-world scenarios, characterized by diverse and unpredictable dynamics, noisy signals, and large state and action spaces, remains limited. This limitation stems from poor data efficiency, limited generalization capabilities, a lack of safety guarantees, and the absence of interpretability, among other factors. To overcome these challenges and improve performance across thesecrucial metrics, one promising avenue is to incorporate additional structural information about the problem into the RL learning process. Various sub-fields of RL have proposed methods for incorporating such inductive biases. We amalgamate these diverse methodologies under a unified framework, shedding light on the role of structure in the learning problem, and classify these methods into distinct patterns of incorporating structure. By leveraging this comprehensive framework, we provide valuable insights into the challenges of structured RL and lay the groundwork for a design pattern perspective on RL research. This novelperspective paves the way for future advancements and aids in developing more effective and efficient RL algorithms that can potentially handle real-world scenarios better.

show abstract

“…The latter is based on the dynamic optimization of the DN according to the load change and the operation constraints of the DN during different phases. In addition, sequential decision-making (SDM) is a time-sequential multi-stage optimization problem, where a controller can interact with system to obtain various sequential decisions (strategies) to maximize gains or minimize losses [6]. The SDM problem is more complex than a series of multiple independent decision problems since the controller considers the long-term effects of its decisions [7].…”

mentioning

confidence: 99%

Sequential Reconfiguration of Unbalanced Distribution Network with Soft Open Points Based on Deep Reinforcement Learning

Yin

Wang

Zhao

2023

Journal of Modern Power Systems and Clean Energy

View full text Add to dashboard Cite

With the large-scale distributed generations (DGs) being connected to distribution network (DN), the traditional day-ahead reconfiguration methods based on physical models are challenged to maintain the robustness and avoid voltage offlimits. To address these problems, this paper develops a deep reinforcement learning method for the sequential reconfiguration with soft open points (SOPs) based on real-time data. A statebased decision model is first proposed by constructing a Marko decision process-based reconfiguration and SOP joint optimization model so that the decisions can be achieved in milliseconds. Then, a deep reinforcement learning joint framework including branching double deep Q network (BDDQN) and multi-policy soft actor-critic (MPSAC) is proposed, which has significantly improved the learning efficiency of the decision model in multidimensional mixed-integer action space. And the influence of DG and load uncertainty on control results has been minimized by using the real-time status of the DN to make control decisions. The numerical simulations on the IEEE 34-bus and 123bus systems demonstrate that the proposed method can effectively reduce the operation cost and solve the overvoltage problem caused by high ratio of photovoltaic (PV) integration.

show abstract

A survey of knowledge‐based sequential decision‐making under uncertainty

Cited by 4 publications

References 80 publications

Advancing autonomous vehicle control systems: An in‐depth overview of decision‐making and manoeuvre execution state of the art

Advancing autonomous vehicle control systems: An in‐depth overview of decision‐making and manoeuvre execution state of the art

Structure in Deep Reinforcement Learning: A Survey and Open Problems

Sequential Reconfiguration of Unbalanced Distribution Network with Soft Open Points Based on Deep Reinforcement Learning

Contact Info

Product

Resources

About