Transfer Learning via Advice Taking

Abstract

The goal of transfer learning is to speed up learning in a new task by transferring knowledge from one or more related source tasks. We describe a transfer method in which a reinforcement learner analyzes its experience in the source task and learns rules to use as advice in the target task. The rules, which are learned via inductive logic programming, describe the conditions under which an action is successful in the source task. The advice-taking algorithm used in the target task allows a reinforcement learner to benefit from rules even if they are imperfect. A human-provided mapping describes the alignment between the source and target tasks, and may also include advice about the differences between them. Using three tasks in the RoboCup simulated soccer domain, we demonstrate that this transfer method can speed up reinforcement learning substantially.

Learning potential functions and their representations for multi-task reinforcement learning

“…Torrey et al [81] identify actions in source tasks with higher Q-values than others, and use this information to construct rules on action preferences that are added as constraints to a linear program for batch-learning Q-function weights in the target task. In [80], this work is extended by using inductive logic programming to extract the rules. Taylor et al [78] learn rules that summarize a learned source task policy and incorporate these as an extra action in the target task, to be learned by a policy-search method.…”

Automatic Face Recognition Based on Sparse Representation and Extended Transfer Learning

“…In this section, we demonstrate the basis of the proposed method. The starting-point method (SPM) is a typical transfer learning algorithm that sets the initial solution in a target task based on knowledge from a source task [27]. Our transfer learning scheme can be viewed as a generalized form of SPM that utilizes the solution of sparse representation on original samples to influence the labeled samples and then obtains the ultimate solution of sparse representation on the labeled samples.…”

“…Z. Liu et al: Automatic Face Recognition Based on Sparse Representation and Extended Transfer Learning learning in a new task by transferring knowledge from a related task that is previously learned [27].…”

Automated transfer in reinforcement learning

“…Human advice aims at integrating humans to an RL agent. For instance, the human can provide action suggestions to the agent [61,111,112] or guide the agent through online feedback [43,45]. In human advice, two major problems standout.…”