Can We Characterize Tasks Without Labels or Features?

Wallace, Bram; Wu, Ziyang; Hariharan, Bharath

doi:10.1109/cvpr46437.2021.00130

Cited by 5 publications

(3 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Neural Tangent Kernels. NTK were first introduced in [18] and developed in subsequent works [31][32][33][34]. NTK employ the gradient of kernels to describe the convergence behavior and to mimic the performance of (over-parameterized) DNN within a limit of infinite width.…”

Section: Related Workmentioning

confidence: 99%

“…NTK employ the gradient of kernels to describe the convergence behavior and to mimic the performance of (over-parameterized) DNN within a limit of infinite width. TTK [34] uses a kernelized distance across the gradients of multiple random initialized networks to estimate the similarity over different tasks. While the ultra-wide networks do not apply to incremental learning, capturing the gradient flow in each incremental learning step could assist in coordinating the corresponding feature representations.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

TKIL: Tangent Kernel Approach for Class Balanced Incremental Learning

Xiang¹,

Shlizerman²

2022

Preprint

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

TKIL: Tangent Kernel Approach for Class Balanced Incremental Learning

Xiang¹,

Shlizerman²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The similarity between tasks is the key point of selecting tasks. Researchers have proposed some methods to measure the distance between tasks including measuring method with probe network [8], [9] and measuring method with distribution [10]. The probe shows the optimal transport distance (or similarity) between 30000 training tasks with another 1 target task.…”

Section: Introductionmentioning

confidence: 99%

Selecting task with optimal transport self-supervised learning for few-shot classification

Xu¹,

Yang²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Few-Shot classification aims at solving problems that only a few samples are available in the training process. Due to the lack of samples, researchers generally employ a set of training tasks from other domains to assist the target task, where the distribution between assistant tasks and the target task is usually different. To reduce the distribution gap, several lines of methods have been proposed, such as data augmentation and domain alignment. However, one common drawback of these algorithms is that they ignore the similarity task selection before training. The fundamental problem is to push the auxiliary tasks close to the target task. In this paper, we propose a novel task selecting algorithm, named Optimal Transport Task Selecting (OTTS), to construct a training set by selecting similar tasks for Few-Shot learning. Specifically, the OTTS measures the task similarity by calculating the optimal transport distance and completes the model training via a self-supervised strategy. By utilizing the selected tasks with OTTS, the training process of Few-Shot learning become more stable and effective. Other proposed methods including data augmentation and domain alignment can be used in the meantime with OTTS. We conduct extensive experiments on a variety of datasets, including MiniImageNet, CIFAR, CUB, Cars, and Places, to evaluate the effectiveness of OTTS. Experimental results validate that our OTTS outperforms the typical baselines, i.e., MAML, matchingnet, protonet, by a large margin (averagely 1.72% accuracy improvement).

show abstract