A Monte-Carlo AIXI Approximation

Veness, Joel; Ng, Kee Siong; Hutter, Marcus; Uther, William; Silver, David

doi:10.1613/jair.3125

Cited by 112 publications

(105 citation statements)

References 72 publications

Supporting

Mentioning

104

Contrasting

Order By: Relevance

“…Summing up, while there has been some work on comparing humans and machines on some specific tasks, e.g., humans and Q-learning in [2], this paper may start a series of experimental research comparing several artificial agents (such as other algorithms in reinforcement learning, MonteCarlo AIXI [16], etc.) and other biological agents (children, other apes, etc) for general tasks.…”

Section: Discussionmentioning

confidence: 99%

Comparing Humans and AI Agents

Insa-Cabrera

Dowe

España

et al. 2011

Artificial General Intelligence

View full text Add to dashboard Cite

Abstract. Comparing humans and machines is one important source of information about both machine and human strengths and limitations. Most of these comparisons and competitions are performed in rather specific tasks such as calculus, speech recognition, translation, games, etc. The information conveyed by these experiments is limited, since it portrays that machines are much better than humans at some domains and worse at others. In fact, CAPTCHAs exploit this fact. However, there have only been a few proposals of general intelligence tests in the last two decades, and, to our knowledge, just a couple of implementations and evaluations. In this paper, we implement one of the most recent test proposals, devise an interface for humans and use it to compare the intelligence of humans and Q-learning, a popular reinforcement learning algorithm. The results are highly informative in many ways, raising many questions on the use of a (universal) distribution of environments, on the role of measuring knowledge acquisition, and other issues, such as speed, duration of the test, scalability, etc.

show abstract

Section: Discussionmentioning

confidence: 99%

Comparing Humans and AI Agents

Insa-Cabrera

Dowe

España

et al. 2011

Artificial General Intelligence

View full text Add to dashboard Cite

show abstract

“…The above can easily be shown (for example, see Proposition 1 in the work of Veness et al (2011)) to define a valid environment model. Because of this, we can simply use…”

Section: Definition 2 Given a Finite Set Of Environment Modelsmentioning

confidence: 98%

“…Note however that in some special cases, more efficient techniques exist with time complexity sublinear in |M|. One example is Context Tree Weighting (Willems et al, 1995), which was used as the basis for our previous AIXI approximations ( Veness et al, 2010Veness et al, , 2011.…”

Section: Definition 2 Given a Finite Set Of Environment Modelsmentioning

confidence: 99%

“…Within reinforcement learning (Sutton and Barto, 1998), some efforts (Veness et al, 2010(Veness et al, , 2011 have recently been made towards approximating AIXI (Hutter, 2005), an optimality notion for general reinforcement learning agents. Impressively, these agents have been shown to be able to learn, from scratch, to play TicTacToe, Pacman, Kuhn Poker, and other simple games by trial and error alone -even the rules of each game were not communicated to the agent.…”

Section: Introductionmentioning

confidence: 99%

“…A more detailed overview of this framework can be found in the work of Hutter (2005) and Veness et al (2011).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

On Ensemble Techniques for AIXI Approximation

Veness

Sunehag

Hutter

2012

Artificial General Intelligence

View full text Add to dashboard Cite

One of the key challenges in AIXI approximation is model class approximation -i.e. how to meaningfully approximate Solomonoff Induction without requiring an infeasible amount of computation? This paper advocates a bottom-up approach to this problem, by describing a number of principled ensemble techniques for approximate AIXI agents. Each technique works by efficiently combining a set of existing environment models into a single, more powerful model. These techniques have the potential to play an important role in future AIXI approximations.

show abstract

Computational models of planning

Geffner

2013

WIRES Cognitive Science

View full text Add to dashboard Cite

The selection of the action to do next is one of the central problems faced by autonomous agents. Natural and artificial systems address this problem in various ways: action responses can be hardwired, they can be learned, or they can be computed from a model of the situation, the actions, and the goals. Planning is the model-based approach to action selection and a fundamental ingredient of intelligent behavior in both humans and machines. Planning, however, is computationally hard as the consideration of all possible courses of action is not computationally feasible. The problem has been addressed by research in Artificial Intelligence that in recent years has uncovered simple but powerful computational principles that make planning feasible. The principles take the form of domain-independent methods for computing heuristics or appraisals that enable the effective generation of goal-directed behavior even over huge spaces. In this paper, we look at several planning models, at methods that have been shown to scale up to large problems, and at what these methods may suggest about the human mind. WIREs Cogn Sci 2013, 4:341-356. doi: 10.1002/wcs.1233 The authors have declared no conflicts of interest for this article. For further resources related to this article, please visit the WIREs website.

show abstract

A Monte-Carlo AIXI Approximation

Cited by 112 publications

References 72 publications

Comparing Humans and AI Agents

Comparing Humans and AI Agents

On Ensemble Techniques for AIXI Approximation

Computational models of planning

Contact Info

Product

Resources

About