Meta-Learning of Neural Architectures for Few-Shot Learning

Elsken, Thomas; Staffler, Benedikt; Metzen, Jan Hendrik; Hutter, Frank

doi:10.1109/cvpr42600.2020.01238

Cited by 111 publications

(55 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Pasunuru et al [295], introduced a continual architecture search (CAS) approach enabling lifelong learning. In addition, as the core idea of auoML is to learn to learn, it is natural to find a growing body of research that combines meta-learning and autoML, particularly for NAS improvement [296,297]. AutoML has also been studied in few-shot learning scenarios, for instance, Elsken et al [297] applied NAS to few-shot learning to overcome the data scarcity, while they only search for the most promising architecture and optimize it to work on multiple few-shot learning tasks.…”

Section: Discussionmentioning

confidence: 99%

“…In addition, as the core idea of auoML is to learn to learn, it is natural to find a growing body of research that combines meta-learning and autoML, particularly for NAS improvement [296,297]. AutoML has also been studied in few-shot learning scenarios, for instance, Elsken et al [297] applied NAS to few-shot learning to overcome the data scarcity, while they only search for the most promising architecture and optimize it to work on multiple few-shot learning tasks. Recently, the idea of unsupervised autoML has begun to be explored, Liu et al [298] proposed a general problem setup, namely unsupervised neural architecture search (UnNAS), to explore whether labels are necessary for NAS.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

A survey on data‐efficient algorithms in big data era

Adadi¹

2021

J Big Data

131

View full text Add to dashboard Cite

The leading approaches in Machine Learning are notoriously data-hungry. Unfortunately, many application domains do not have access to big data because acquiring data involves a process that is expensive or time-consuming. This has triggered a serious debate in both the industrial and academic communities calling for more data-efficient models that harness the power of artificial learners while achieving good results with less training data and in particular less human supervision. In light of this debate, this work investigates the issue of algorithms’ data hungriness. First, it surveys the issue from different perspectives. Then, it presents a comprehensive review of existing data-efficient methods and systematizes them into four categories. Specifically, the survey covers solution strategies that handle data-efficiency by (i) using non-supervised algorithms that are, by nature, more data-efficient, by (ii) creating artificially more data, by (iii) transferring knowledge from rich-data domains into poor-data domains, or by (iv) altering data-hungry algorithms to reduce their dependency upon the amount of samples, in a way they can perform well in small samples regime. Each strategy is extensively reviewed and discussed. In addition, the emphasis is put on how the four strategies interplay with each other in order to motivate exploration of more robust and data-efficient algorithms. Finally, the survey delineates the limitations, discusses research challenges, and suggests future opportunities to advance the research on data-efficiency in machine learning.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

A survey on data‐efficient algorithms in big data era

Adadi¹

2021

J Big Data

131

View full text Add to dashboard Cite

show abstract

“…Meta-training across multiple datasets may lead to improved cross-task generalization of architectures [131]. Finally, one can also define NAS meta-objectives to train an architecture suitable for few-shot learning [243]. Similarly to fast-adapting initial condition meta-learning approaches such as MAML [19], one can train good initial architectures [130] or architecture priors [131] that are easy to adapt towards specific tasks.…”

Section: Neural Architecture Search (Nas)mentioning

confidence: 99%

Meta-Learning in Neural Networks: A Survey

Hospedales¹,

Antoniou²,

Micaelli³

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

604

274

View full text Add to dashboard Cite

The field of meta-learning, or learning-to-learn, has seen a dramatic rise in interest in recent years. Contrary to conventional approaches to AI where tasks are solved from scratch using a fixed learning algorithm, meta-learning aims to improve the learning algorithm itself, given the experience of multiple learning episodes. This paradigm provides an opportunity to tackle many conventional challenges of deep learning, including data and computation bottlenecks, as well as generalization. This survey describes the contemporary meta-learning landscape. We first discuss definitions of meta-learning and position it with respect to related fields, such as transfer learning and hyperparameter optimization. We then propose a new taxonomy that provides a more comprehensive breakdown of the space of meta-learning methods today. We survey promising applications and successes of meta-learning such as few-shot learning and reinforcement learning. Finally, we discuss outstanding challenges and promising areas for future research.

show abstract

“…Currently, active research areas include the use of metalearning techniques to improve the sample efficiency of AutoML-based approaches [212,213], development of better search strategies [142,143] and computationally efficient techniques for model generation [214,215]. In the foreseeable future, however, better NAS algorithms and improvements in hardware performance-especially cloud-based computational resources-will enable the development of NAS techniques that can be used in challenging domains to solve challenging machine vision problems.…”

Section: Emerging Trends and Future Research Directionsmentioning

confidence: 99%

CNN Architectures for Geometric Transformation-Invariant Feature Representation in Computer Vision: A Review

Mumuni

2021

SN COMPUT. SCI.

View full text Add to dashboard Cite

One of the main challenges in machine vision relates to the problem of obtaining robust representation of visual features that remain unaffected by geometric transformations. This challenge arises naturally in many practical machine vision tasks. For example, in mobile robot applications like simultaneous localization and mapping (SLAM) and visual tracking, object shapes change depending on their orientation in the 3D world, camera proximity, viewpoint, or perspective. In addition, natural phenomena such as occlusion, deformation, and clutter can cause geometric appearance changes of the underlying objects, leading to geometric transformations of the resulting images. Recently, deep learning techniques have proven very successful in visual recognition tasks but they typically perform poorly with small data or when deployed in environments that deviate from training conditions. While convolutional neural networks (CNNs) have inherent representation power that provides a high degree of invariance to geometric image transformations, they are unable to satisfactorily handle nontrivial transformations. In view of this limitation, several techniques have been devised to extend CNNs to handle these situations. This article reviews some of the most promising approaches to extend CNN architectures to handle nontrivial geometric transformations. Key strengths and weaknesses, as well as the application domains of the various approaches are also highlighted. The review shows that although an adequate model for generalized geometric transformations has not yet been formulated, several techniques exist for solving specific problems. Using these methods, it is possible to develop task-oriented solutions to deal with nontrivial transformations.

show abstract

Meta-Learning of Neural Architectures for Few-Shot Learning

Cited by 111 publications

References 14 publications

A survey on data‐efficient algorithms in big data era

A survey on data‐efficient algorithms in big data era

Meta-Learning in Neural Networks: A Survey

CNN Architectures for Geometric Transformation-Invariant Feature Representation in Computer Vision: A Review

Contact Info

Product

Resources

About