Optimal Ordered Problem Solver

Schmidhuber, Jürgen

doi:10.1023/b:mach.0000015880.99707.b2

Cited by 115 publications

(171 citation statements)

References 63 publications

Supporting

Mentioning

168

Contrasting

Order By: Relevance

“…Often it is much more efficient to systematically search the space of such programs with a bias towards short and fast programs (Levin, 1973b;Schmidhuber, 1997Schmidhuber, , 2004, instead of directly searching the huge space of possible NN weight matrices. A previous universal language for encoding NNs was assembler-like (Schmidhuber, 1997).…”

Section: Deep Hierarchical Rl (Hrl) and Subgoal Learning With Fnns Anmentioning

confidence: 99%

Deep learning in neural networks: An overview

2015

Self Cite

View full text Add to dashboard Cite

In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarises relevant work, much of it from the previous millennium. Shallow and deep learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks.LATEX source: http://www.idsia.ch/˜juergen/DeepLearning8Oct2014.tex Complete BIBTEX file (888 kB): http://www.idsia.ch/˜juergen/deep.bib Preface This is the preprint of an invited Deep Learning (DL) overview. One of its goals is to assign credit to those who contributed to the present state of the art. I acknowledge the limitations of attempting to achieve this goal. The DL research community itself may be viewed as a continually evolving, deep network of scientists who have influenced each other in complex ways. Starting from recent DL results, I tried to trace back the origins of relevant ideas through the past half century and beyond, sometimes using "local search" to follow citations of citations backwards in time. Since not all DL publications properly acknowledge earlier relevant work, additional global search strategies were employed, aided by consulting numerous neural network experts. As a result, the present preprint mostly consists of references. Nevertheless, through an expert selection bias I may have missed important work. A related bias was surely introduced by my special familiarity with the work of my own DL research group in the past quarter-century. For these reasons, this work should be viewed as merely a snapshot of an ongoing credit assignment process. To help improve it, please do not hesitate to send corrections and suggestions to juergen@idsia.ch.

show abstract

Section: Deep Hierarchical Rl (Hrl) and Subgoal Learning With Fnns Anmentioning

confidence: 99%

Deep learning in neural networks: An overview

2015

Self Cite

View full text Add to dashboard Cite

show abstract

“…Learning at the metalevel is concerned with accumulating experience on the performance of multiple applications of a learning system. A more integrated approach resembling meta-learning and incremental learning is [25], where the authors present the Optimal Ordered Problem Solver (OOPS), an optimally fast way of incrementally solving each task in the sequence by reusing successful code from previous tasks.…”

Section: Previous Workmentioning

confidence: 99%

Learning with Configurable Operators and RL-Based Heuristics

Martínez-Plumed

Ferri

Hernández-Orallo

et al. 2013

New Frontiers in Mining Complex Patterns

View full text Add to dashboard Cite

Abstract. In this paper, we push forward the idea of machine learning systems for which the operators can be modified and finetuned for each problem. This allows us to propose a learning paradigm where users can write (or adapt) their operators, according to the problem, data representation and the way the information should be navigated. To achieve this goal, data instances, background knowledge, rules, programs and operators are all written in the same functional language, Erlang. Since changing operators affect how the search space needs to be explored, heuristics are learnt as a result of a decision process based on reinforcement learning where each action is defined as a choice of operator and rule. As a result, the architecture can be seen as a 'system for writing machine learning systems' or to explore new operators.

show abstract

“…The down-scaled still provably optimal AIXItl model [Hut05,Chp.7] based on universal search algorithms [Lev73,Hut02a,Gag07] was still computationally intractable. The Optimal Ordered Problem Solver [Sch04] was the first practical implementation of universal search and has been able to solve open learning tasks such as Towers-of-Hanoi for arbitrary number of disks, robotic behavior, and others.…”

Section: Theory Of Uaimentioning

confidence: 99%

One Decade of Universal Artificial Intelligence

Hutter¹

2012

Atlantis Thinking Machines

View full text Add to dashboard Cite

The first decade of this century has seen the nascency of the first mathematical theory of general artificial intelligence. This theory of Universal Artificial Intelligence (UAI) has made significant contributions to many theoretical, philosophical, and practical AI questions. In a series of papers culminating in book (Hutter, 2005), an exciting sound and complete mathematical model for a super intelligent agent (AIXI) has been developed and rigorously analyzed. While nowadays most AI researchers avoid discussing intelligence, the awardwinning PhD thesis (Legg, 2008) provided the philosophical embedding and investigated the UAI-based universal measure of rational intelligence, which is formal, objective and non-anthropocentric. Recently, effective approximations of AIXI have been derived and experimentally investigated in JAIR paper ). This practical breakthrough has resulted in some impressive applications, finally muting earlier critique that UAI is only a theory. For the first time, without providing any domain knowledge, the same agent is able to self-adapt to a diverse range of interactive environments. For instance, AIXI is able to learn from scratch to play TicTacToe, Pacman, Kuhn Poker, and other games by trial and error, without even providing the rules of the games.These achievements give new hope that the grand goal of Artificial General Intelligence is not elusive.This article provides an informal overview of UAI in context. It attempts to gently introduce a very theoretical, formal, and mathematical subject, and discusses philosophical and technical ingredients, traits of intelligence, some social questions, and the past and future of UAI.

show abstract

Optimal Ordered Problem Solver

Cited by 115 publications

References 63 publications

Deep learning in neural networks: An overview

Deep learning in neural networks: An overview

Learning with Configurable Operators and RL-Based Heuristics

One Decade of Universal Artificial Intelligence

Contact Info

Product

Resources

About