Combined Reinforcement Learning via Abstract Representations

François-Lavet, Vincent; Bengio, Yoshua; Precup, Doina; Pineau, Joëlle

doi:10.1609/aaai.v33i01.33013582

Cited by 53 publications

(63 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The CRAR agent explicitly learns both a value function and a model via a shared low-dimensional learned encoding of the environment, which is meant to capture summarized abstractions and allow for efficient planning (François-Lavet et al, 2018). By forcing an expressive representation, the CRAR approach creates an interpretable low-dimensional representation of the environment, even far temporally from any rewards or in the absence of model-free objectives.…”

Section: Value-based Rlmentioning

confidence: 99%

“…In deep RL, it is possible to build an abstract state such that it provides sufficient information for simultaneously fitting an internal meaningful dynamics as well as the estimation of the expected value of an optimal policy. By explicitly learning both the model-free and model-based components through the state representation, along with an approximate entropy maximization penalty, the CRAR agent (François-Lavet et al, 2018) shows how it is possible to learn a low-dimensional representation of the task. In addition, this approach can directly make use of a combination of model-free and model-based, with planning happening in a smaller latent state space.…”

Section: Auxiliary Tasksmentioning

confidence: 99%

“…Rather than manually tuning the randomization of simulations, one can also adapt the simulation parameters by matching the policy behavior in simulation to the real world (Chebotar et al, 2018). Another approach to zero-shot transfer is to use algorithms that enforce states that relate to the same underlying task but have different renderings to be mapped into an abstract state that is close (Tzeng et al, 2015;François-Lavet et al, 2018).…”

Section: Zero-shot Learningmentioning

confidence: 99%

See 2 more Smart Citations

An Introduction to Deep Reinforcement Learning

François-Lavet

Henderson

Islam

et al. 2018

FNT in Machine Learning

Self Cite

633

197

View full text Add to dashboard Cite

Deep reinforcement learning is the combination of reinforcement learning (RL) and deep learning. This field of research has been able to solve a wide range of complex decisionmaking tasks that were previously out of reach for a machine. Thus, deep RL opens up many new applications in domains such as healthcare, robotics, smart grids, finance, and many more. This manuscript provides an introduction to deep reinforcement learning models, algorithms and techniques. Particular focus is on the aspects related to generalization and how deep RL can be used for practical applications. We assume the reader is familiar with basic machine learning concepts. may be may be constrained (e.g., not access to an accurate simulator or limited data).Over the past few years, RL has become increasingly popular due to its success in addressing challenging sequential decision-making problems. Several of these achievements are due to the combination of RL with deep learning techniques (LeCun et al., 2015;Schmidhuber, 2015;Goodfellow et al., 2016). This combination, called deep RL, is most useful in problems with high dimensional state-space. Previous RL approaches had a difficult design issue in the choice of features (Munos and Moore, 2002;Bellemare et al., 2013). However, deep RL has been successful in complicated tasks with lower prior knowledge thanks to its ability to learn different levels of abstractions from data. For instance, a deep RL agent can successfully learn from visual perceptual inputs made up of thousands of pixels (Mnih et al., 2015). This opens up the possibility to mimic some human problem solving capabilities, even in high-dimensional space -which, only a few years ago, was difficult to conceive.Several notable works using deep RL in games have stood out for attaining super-human level in playing Atari games from the pixels (

show abstract

Section: Value-based Rlmentioning

confidence: 99%

Section: Auxiliary Tasksmentioning

confidence: 99%

Section: Zero-shot Learningmentioning

confidence: 99%

See 1 more Smart Citation

An Introduction to Deep Reinforcement Learning

François-Lavet

Henderson

Islam

et al. 2018

FNT in Machine Learning

Self Cite

633

197

View full text Add to dashboard Cite

show abstract

“…The recently introduced Consciousness Prior (CP; Bengio, 2017) is a framework to represent the mental model of a single agent, through the notion of abstract state representations. 6 Here, an abstract state corresponds withs ( §4), a low-dimensional, structured, interpretable state encoding, useful for planning, communication, and predicting upcoming observations (François-Lavet et al, 2019). One example is a dynamic knowledge graph embedding to represent a scene (Kipf et al, 2020).…”

Section: Formal Frameworkmentioning

confidence: 99%

Language (Re)modelling: Towards Embodied Language Understanding

Tamari

Shani

Hope

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

While natural language understanding (NLU) is advancing rapidly, today's technology differs from human-like language understanding in fundamental ways, notably in its inferior efficiency, interpretability, and generalization. This work proposes an approach to representation and learning based on the tenets of embodied cognitive linguistics (ECL). According to ECL, natural language is inherently executable (like programming languages), driven by mental simulation and metaphoric mappings over hierarchical compositions of structures and schemata learned through embodied interaction. This position paper argues that the use of grounding by metaphoric inference and simulation will greatly benefit NLU systems, and proposes a system architecture along with a roadmap towards realizing this vision.

show abstract

“…FEWS integrated management requires a combination of economic-based management strategy evaluation, with optimization that incorporates environmental impacts and risk of climate change. Decision theory and reinforcement learning make this integration possible; recent advancements in these fields have shown great promise in modeling complex dynamics of interdependent systems (Littman 2015) in many real-world applications such as human-level control in gaming (Mnih et al 2015, Silver et al 2017, natural resource management Boettiger 2018, 2019), and robotics (Porta et al 2005, Francois-Lavet et al 2018. In this article we develop a dynamic optimization approach basing upon fundamentals of decision theory and model-based reinforcement learning, to adaptively control and optimize operation of integrated FEWS.…”

Section: Introductionmentioning

confidence: 99%

Optimizing dynamics of integrated food–energy–water systems under the risk of climate change

Memarzadeh

Moura

Horvath

2019

Environ. Res. Lett.

View full text Add to dashboard Cite

Integrated management of food-energy-water systems (FEWS) requires a unified, flexible and reproducible approach to incorporate the interdependence between sectors, and include the risk of non-stationary environmental variations due to climate change. Most of the recently developed methods in the literature fall short of one or more aspects in such integration. In this article, we propose a novel approach based upon fundamentals of decision theory and reinforcement learning that (1) quantifies and propagates uncertainty, (2) incorporates resource interdependence, (3) includes the impact of uncontrolled variables such as climate variations, and (4) adaptively optimizes management decisions to minimize the costs and environmental impacts of crop production. Moreover, the proposed method is robust to problem-specific complexities and is easily reproducible. We illustrate the framework on a real-world case study in Ventura County, California.

show abstract

Combined Reinforcement Learning via Abstract Representations

Cited by 53 publications

References 2 publications

An Introduction to Deep Reinforcement Learning

An Introduction to Deep Reinforcement Learning

Language (Re)modelling: Towards Embodied Language Understanding

Optimizing dynamics of integrated food–energy–water systems under the risk of climate change

Contact Info

Product

Resources

About