Meta-Learning in Neural Networks: A Survey

Hospedales, Timothy M.; Antoniou, Antreas; Micaelli, Paul; Storkey, Amos

doi:10.48550/arxiv.2004.05439

Cited by 208 publications

(287 citation statements)

References 0 publications

Supporting

Mentioning

276

Contrasting

Order By: Relevance

“…Meta Learning. Meta-learning [14][26] [31][34] [35], or learning to learn, is the science of systematically observing how different machine learning approaches perform in multiple tasks, and learning from this experience to seek rapid and accurate model adaptation to unseen tasks, even with few training examples. It has been widely used in many applications like highly automated AI, few-shot learning, natural language processing [32] and image processing [6] [23].…”

Section: Related Workmentioning

confidence: 99%

Leaving No One Behind: A Multi-Scenario Multi-Task Meta Learning Approach for Advertiser Modeling

Zhang

Liao

Liu

et al. 2022

Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

View full text Add to dashboard Cite

Advertisers play an essential role in many e-commerce platforms like Taobao and Amazon. Fulfilling their marketing needs and supporting their business growth is critical to the long-term prosperity of platform economies. However, compared with extensive studies on user modeling such as click-through rate predictions, much less attention has been drawn to advertisers, especially in terms of understanding their diverse demands and performance. Different from user modeling, advertiser modeling generally involves many kinds of tasks (e.g. predictions of advertisers' expenditure, active-rate, or total impressions of promoted products). In addition, major e-commerce platforms often provide multiple marketing scenarios (e.g. Sponsored Search, Display Ads, Live Streaming Ads) while advertisers' behavior tend to be dispersed among many of them. This raises the necessity of multi-task and multi-scenario consideration in comprehensive advertiser modeling, which faces the following challenges: First, one model per scenario or per task simply doesn't scale; Second, it is particularly hard to model new or minor scenarios with limited data samples; Third, inter-scenario correlations are complicated, and may vary given different tasks.To tackle these challenges, we propose a multi-scenario multitask meta learning approach (M2M) which simultaneously predicts multiple tasks in multiple advertising scenarios. Specifically, we introduce a novel meta unit that incorporates rich scenario knowledge to learn explicit inter-scenario correlations and can easily scale to new scenarios. Furthermore, we present a meta attention module to capture diverse inter-scenario correlations given different tasks, and a meta tower module to enhance the capability of capturing the representation of scenario-specific features. Compelling results from both offline evaluation and online A/B tests demonstrate the superiority of M2M over state-of-the-art methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Leaving No One Behind: A Multi-Scenario Multi-Task Meta Learning Approach for Advertiser Modeling

Zhang

Liao

Liu

et al. 2022

Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

View full text Add to dashboard Cite

show abstract

“…However, traditional machine learning systems do not perform well under such constraints. Meta-learning approaches tackle this problem, creating a general learner that is able to adapt to a new task with a small number of samples [15]. Previous work in meta-learning had focused on supervised computer vision tasks [37,49] and applied these methods to image analysis [20,44].…”

Section: Meta-learning and Personalized Physiological Sensingmentioning

confidence: 99%

“…Meta learning is an emerging technique in machine learning that aims to learn how to learn a task faster [15]. The goal of meta learning is to learn a quick learner for a new task (e.g., person).…”

Section: Introductionmentioning

confidence: 99%

MobilePhys: Personalized Mobile Camera-Based Contactless Physiological Sensing

Liu,

Wang,

Xie

et al. 2022

Preprint

View full text Add to dashboard Cite

Camera-based contactless photoplethysmography refers to a set of popular techniques for contactless physiological measurement. The current state-of-the-art neural models are typically trained in a supervised manner using videos accompanied by gold standard physiological measurements. However, they often generalize poorly out-of-domain examples (i.e., videos that are unlike those in the training set). Personalizing models can help improve model generalizability, but many personalization techniques still require some gold standard data. To help alleviate this dependency, in this paper, we present a novel mobile sensing system called MobilePhys, the first mobile personalized remote physiological sensing system, that leverages both front and rear cameras on a smartphone to generate high-quality self-supervised labels for training personalized contactless camera-based PPG models. To evaluate the robustness of MobilePhys, we conducted a user study with 39 participants who completed a set of tasks under different mobile devices, lighting conditions/intensities, motion tasks, and skin types. Our results show that MobilePhys significantly outperforms the state-of-the-art on-device supervised training and few-shot adaptation methods. Through extensive user studies, we further examine how does MobilePhys perform in complex real-world settings. We envision that calibrated or personalized camera-based contactless PPG models generated from our proposed dual-camera mobile sensing system will open the door for numerous future applications such as smart mirrors, fitness and mobile health applications.

show abstract

“…Parallel learning shares a portion of the neurons across multiple tasks and trains them on those tasks simultaneously whereas in transfer learning, neurons are initialized with values learned on an earlier task and are trained on a new task. Another group of algorithms called Meta-learning [26,27,28] aim to reduce the required amount of data to learn a new task using trained models on other tasks. Although meta-learning algorithms have been proposed to learn tasks with heterogeneous attribute spaces [37], these methods are yet to be extended to heterogeneous tasks.…”

Section: Related Workmentioning

confidence: 99%

PaRT: Parallel Learning Towards Robust and Transparent AI

Paknezhad¹,

Rengarajan²,

Yuan³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper takes a parallel learning approach for robust and transparent AI. A deep neural network is trained in parallel on multiple tasks, where each task is trained only on a subset of the network resources. Each subset consists of network segments, that can be combined and shared across specific tasks. Tasks can share resources with other tasks, while having independent task-related network resources. Therefore, the trained network can share similar representations across various tasks, while also enabling independent task-related representations. The above allows for some crucial outcomes. (1) The parallel nature of our approach negates the issue of catastrophic forgetting. (2) The sharing of segments uses network resources more efficiently. (3) We show that the network does indeed use learned knowledge from some tasks in other tasks, through shared representations. (4) Through examination of individual task-related and shared representations, the model offers transparency in the network and in the relationships across tasks in a multi-task setting. Evaluation of the proposed approach against complex competing approaches such as Continual Learning, Neural Architecture Search, and Multi-task learning shows that it is capable of learning robust representations. This is the first effort to train a DL model on multiple tasks in parallel. Our code is available at https://github.com/MahsaPaknezhad/PaRT.

show abstract

Meta-Learning in Neural Networks: A Survey

Cited by 208 publications

References 0 publications

Leaving No One Behind: A Multi-Scenario Multi-Task Meta Learning Approach for Advertiser Modeling

Leaving No One Behind: A Multi-Scenario Multi-Task Meta Learning Approach for Advertiser Modeling

MobilePhys: Personalized Mobile Camera-Based Contactless Physiological Sensing

PaRT: Parallel Learning Towards Robust and Transparent AI

Contact Info

Product

Resources

About