Auto-Lambda: Disentangling Dynamic Task Relationships

Abstract

Understanding the structure of multiple related tasks allows for multi-task learning to improve the generalisation ability of one or all of them. However, it usually requires training each pairwise combination of tasks together in order to capture task relationships, at an extremely high computational cost. In this work, we learn task relationships via an automated weighting framework, named Auto-λ. Unlike previous methods where task relationships are assumed to be fixed, Auto-λ is a gradient-based meta learning framework which explores continuous, dynamic task relationships via task-specific weightings, and can optimise any choice of combination of tasks through the formulation of a meta-loss; where the validation loss automatically influences task weightings throughout training. We apply the proposed framework to both multi-task and auxiliary learning problems in computer vision and robotics, and show that Auto-λ achieves state-of-the-art performance, even when compared to optimisation strategies designed specifically for each problem and data domain. Finally, we observe that Auto-λ can discover interesting learning behaviors, leading to new insights in multi-task learning. Code is available at https://github.com/lorenmt/auto-lambda.

Curriculum-Based Asymmetric Multi-Task Reinforcement Learning

“…To show the capability of incorporating prior knowledge for CAMRL, we perform extra experiments that formulate a new differentiable ranking loss for tasks where the relative magnitudes of the task difficulty are readily apparent in part. Specifically, we follow [36], [36], [26], [37], and [21] to obtain the public performance of existing state-of-the-arts methods for MT10, MT50, Atari, Ravens, and RLBench, respectively. Then we use the relative ranking of the public performance to formulate a new tanh-based differentiable ranking loss and incorporate it with Eq.…”

“…RLBench: RLBench is a large-scale environment designed to speed up vision-guided manipulation research. We follow [21]…”