Markov decision processes

Altman, Eitan

doi:10.1201/9781315140223-3

Cited by 257 publications

(489 citation statements)

References 0 publications

Supporting

Mentioning

486

Contrasting

Order By: Relevance

“…An overview of AI safety methods can be found in Pecka and Svoboda (2014) and Garcıa and Fernández (2015). A large body of work in the area of Safe AI focuses on constrained RL (Altman, 1999;Wen and Topcu, 2018). Constrained RL depends on a-priori defined safety constraints which are states or actions that the agent should avoid.…”

Section: Ai Safetymentioning

confidence: 99%

Safer Reinforcement Learning through Transferable Instinct Networks

Grbic

Risi

2021

The 2021 Conference on Artificial Life

View full text Add to dashboard Cite

Random exploration is one of the main mechanisms through which reinforcement learning (RL) finds well-performing policies. However, it can lead to undesirable or catastrophic outcomes when learning online in safety-critical environments. In fact, safe learning is one of the major obstacles towards real-world agents that can learn during deployment. One way of ensuring that agents respect hard limitations is to explicitly configure boundaries in which they can operate. While this might work in some cases, we do not always have clear a-priori information which states and actions can lead dangerously close to hazardous states. Here, we present an approach where an additional policy can override the main policy and offer a safer alternative action. In our instinctregulated RL (IR 2 L) approach, an "instinctual" network is trained to recognize undesirable situations, while guarding the learning policy against entering them. The instinct network is pre-trained on a single task where it is safe to make mistakes, and transferred to environments in which learning a new task safely is critical. We demonstrate IR 2 L in the Ope-nAI Safety gym domain, in which it receives a significantly lower number of safety violations during training than a baseline RL approach while reaching similar task performance.

show abstract

Section: Ai Safetymentioning

confidence: 99%

Safer Reinforcement Learning through Transferable Instinct Networks

Grbic

Risi

2021

The 2021 Conference on Artificial Life

View full text Add to dashboard Cite

show abstract

“…Stochastic Optimization using Markov Decision Processes has very rich roots (Howard, 1960). There have been work in understanding convergence of the algorithm to find optimal policies for known MDPs (Bertsekas and Tsitsiklis, 1996;Altman, 1999). Also, when the MDP is not known, there are algorithms with asymptotic guarantees for learning the optimal policies (Watkins and Dayan, 1992) which maximize an objective without any constraints.…”

Section: Related Workmentioning

confidence: 99%

“…Similar to the example above, many applications require to keep some costs low while simultaneously maximizing the rewards (Altman, 1999). Owing to the importance of this problem, in this paper, we consider the problem of constrained Markov Decision Processes (constrained MDP or CMDP).…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…The problem setup, where the system dynamics are known, is extensively studied (Altman, 1999). For a constrained setup, the optimal policy is possibly stochastic (Altman, 1999;Puterman, 2014). In the domain where the agent learns the system dynamics and aims to learn good policies online, there has been work where to show asymptotic convergence to optimal policies (Gattami et al, 2021), or even provide regret guarantees when the MDP is episodic (Zheng and Ratliff, 2020;Ding et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Markov Decision Processes with Long-Term Average Constraints

Agarwal¹,

Bai²,

Aggarwal³

2021

Preprint

View full text Add to dashboard Cite

We consider the problem of constrained Markov Decision Process (CMDP) where an agent interacts with a unichain Markov Decision Process. At every interaction, the agent obtains a reward. Further, there are K cost functions. The agent aims to maximize the long-term average reward while simultaneously keeping the K long-term average costs lower than a certain threshold. In this paper, we propose CMDP-PSRL, a posterior sampling based algorithm using which the agent can learn optimal policies to interact with the CMDP. Further, for MDP with S states, A actions, and diameter D, we prove that following CMDP-PSRL algorithm, the agent can bound the regret of not accumulating rewards from optimal policy by Õ(poly(DSA) √T ). Further, we show that the violations for any of the K constraints is also bounded by Õ(poly(DSA) √T ). To the best of our knowledge, this is the first work which obtains a Õ( √ T ) regret bounds for ergodic MDPs with long-term average constraints.

show abstract

Cognitive Internet of Vehicles and disaster management: A proposed architecture and future direction

Arooj

Farooq

Umer

et al. 2019

Trans Emerging Tel Tech

View full text Add to dashboard Cite

Historically, the vehicle has just been a component of the human ambulatory system and slave to the commands of driving force. However, recent advancement in technologies such as 5G wireless systems, cloud/edge computing, machine learning, artificial intelligence, and deep learning have opened the new paradigm of the Cognitive Internet of Vehicles (CIoV). The network of heterogeneous intelligent vehicles, not only having the social interaction capabilities but also have the ability to visualize, capture, and disseminate information to dynamic cognitive engines for identifying and analyzing different patterns and prediction of optimized outcomes. From secure navigation to traffic control, transportation security for pollution control has advocated plausible applications in real-time operations of Internet of Vehicles in humanitarian operations.Proceeding this research, this paper aimed to propose a state-of-the-art architecture based on CIoV for identifying emerging capabilities of available technologies to observe, detect, and mitigate natural disasters, and to develop the CIoV ecosystem for processing the global scale data for intelligent decision making and better service provisioning in natural disaster management.

show abstract

Markov decision processes

Cited by 257 publications

References 0 publications

Safer Reinforcement Learning through Transferable Instinct Networks

Safer Reinforcement Learning through Transferable Instinct Networks

Markov Decision Processes with Long-Term Average Constraints

Cognitive Internet of Vehicles and disaster management: A proposed architecture and future direction

Contact Info

Product

Resources

About