Connecting Context-specific Adaptation in Humans to Meta-learning

Abstract

Cognitive control, the ability of a system to adapt to the demands of a task, is an integral part of cognition. A widely accepted fact about cognitive control is that it is contextsensitive: Adults and children alike infer information about a task's demands from contextual cues and use these inferences to learn from ambiguous cues. However, the precise way in which people use contextual cues to guide adaptation to a new task remains poorly understood. This work connects the context-sensitive nature of cognitive control to a method for meta-learning with context-conditioned adaptation. We begin by identifying an essential difference between human learning and current approaches to meta-learning: In contrast to humans, existing meta-learning algorithms do not make use of task-specific contextual cues but instead rely exclusively on online feedback in the form of task-specific labels or rewards. To remedy this, we introduce a framework for using contextual information about a task to guide the initialization of task-specific models before adaptation to online feedback. We show how context-conditioned meta-learning can capture human behavior in a cognitive task and how it can be scaled to improve the speed of learning in various settings, including few-shot classification and low-sample reinforcement learning. Our work demonstrates that guiding meta-learning with task information can capture complex, human-like behavior, thereby deepening our understanding of cognitive control.

Why context should matter.

“…In recent years, contextual information has been increasingly used to overcome this challenge and improve the state of the art in reinforcement learning (Hallak et al, 2015;Modi et al, 2018;Sodhani et al, 2021). For instance, context-based meta-learning methods allow faster learning of new tasks by efficiently extracting metaknowledge from previously encountered tasks (Chen et al, 2021;Dubey et al, 2020;He et al, 2019;Zintgraf et al, 2019). Here, context serves as an additional input to the model, allowing the model to use contextual information to adapt to individual tasks while the meta-trained parameters are used to learn task-general properties.…”