Adaptation Algorithm and Theory Based on Generalized Discrepancy

Cortes, Corinna; Mohri, Mehryar; Medina, Andrés Muñoz

doi:10.1145/2783258.2783368

Cited by 76 publications

(85 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Adaptive active learning strategies use label information to choose queries, this additional information may lead to improved performance. For domain adaptation, [6] considers such an adaptive approach to improve upon the Discrepancy. Their bound is, however, not trivial to apply to active learning because it is intrinsically designed for domain adaptation.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Nuclear discrepancy for single-shot batch active learning

Viering¹,

Krijthe

Loog³

2019

Mach Learn

View full text Add to dashboard Cite

Active learning algorithms propose which unlabeled objects should be queried for their labels to improve a predictive model the most. We study active learners that minimize generalization bounds and uncover relationships between these bounds that lead to an improved approach to active learning. In particular we show the relation between the bound of the state-of-the-art Maximum Mean Discrepancy (MMD) active learner, the bound of the Discrepancy, and a new and looser bound that we refer to as the Nuclear Discrepancy bound. We motivate this bound by a probabilistic argument: we show it considers situations which are more likely to occur. Our experiments indicate that active learning using the tightest Discrepancy bound performs the worst in terms of the squared loss. Overall, our proposed loosest Nuclear Discrepancy generalization bound performs the best. We confirm our probabilistic argument empirically: the other bounds focus on more pessimistic scenarios that are rarer in practice. We conclude that tightness of bounds is not always of main importance and that active learning methods should concentrate on realistic scenarios in order to improve performance.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Discrepancy Bound for Active Learning. We give a bound of Cortes et al [6] in terms of the Discrepancy. The Discrepancy is defined as…”

Section: Theoretical Analysis Of Existing Boundsmentioning

confidence: 96%

Nuclear discrepancy for single-shot batch active learning

Viering¹,

Krijthe

Loog³

2019

Mach Learn

View full text Add to dashboard Cite

show abstract

“…Even if this optimization task is still not convex (Φ d is quasiconcave), our empirical study shows no need to perform many restarts while performing gradient descent to find a suitable solution. 14 We name this domain adaptation algorithm pbda.…”

Section: Generalization Bounds and Learning Algorithms 6 Pac-bayesianmentioning

confidence: 99%

PAC-Bayes and domain adaptation

et al. 2020

View full text Add to dashboard Cite

We provide two main contributions in PAC-Bayesian theory for domain adaptation where the objective is to learn, from a source distribution, a well-performing majority vote on a different, but related, target distribution. Firstly, we propose an improvement of the previous approach we proposed in [1], which relies on a novel distribution pseudodistance based on a disagreement averaging, allowing us to derive a new tighter domain adaptation bound for the target risk. While this bound stands in the spirit of common domain adaptation works, we derive a second bound (introduced in [2]) that brings a new perspective on domain adaptation by deriving an upper bound on the target risk where the distributions' divergence-expressed as a ratio-controls the trade-off between a source error measure and the target voters' disagreement. We discuss and compare both results, from which we obtain PAC-Bayesian generalization bounds. Furthermore, from the PAC-Bayesian specialization to linear classifiers, we infer two learning algorithms, and we evaluate them on real data.

show abstract

“…All of the aforementioned approaches solve the domain adaptation problem in a classification task, whereas primarily leaf counting is a regression task. A recent work addressing domain adaptation on regression task is described in [4]. The authors propose a convex optimisation framework for sample re-weighting.…”

Section: Domain Adaptationmentioning

confidence: 99%

Leaf Counting Without Annotations Using Adversarial Unsupervised Domain Adaptation

Giuffrida

Dobrescu

Doerner

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

Deep learning is making strides in plant phenotyping and agriculture. But pretrained models require significant adaptation to work on new target datasets originating from a different experiment even on the same species. The current solution is to retrain the model on the new target data implying the need for annotated and labelled images. This paper addresses the problem of adapting a previously trained model on new target but unlabelled images. Our method falls in the broad machine learning problem of domain adaptation, where our aim is to reduce the difference between the source and target dataset (domains). Most classical approaches necessitate that both source and target data are simultaneously available to solve the problem. In agriculture it is possible that source data cannot be shared. Hence, we propose to update the model without necessarily sharing the data of the training source to preserve confidentiality. Our major contribution is a model that reduces the domain shift using an unsupervised adversarial adaptation mechanism on statistics of the training (source) data. In addition, we propose a multi-output training process that (i) allows (quasi-)integer leaf counting predictions; and (ii) improves the accuracy on the target domain, by minimising the distance between the counting distributions on the source and target domain. In our experiments we used a reduced version of the CVPPP dataset as source domain. We performed two sets of experiments, showing domain adaptation in the intra-and inter-species case. Using an Arabidopsis dataset as target domain, the prediction results exhibit a mean squared error (MSE) of 2.3. When a different plant species was used (Komatsuna), the MSE was 1.8.

show abstract

Adaptation Algorithm and Theory Based on Generalized Discrepancy

Cited by 76 publications

References 20 publications

Nuclear discrepancy for single-shot batch active learning

Nuclear discrepancy for single-shot batch active learning

PAC-Bayes and domain adaptation

Leaf Counting Without Annotations Using Adversarial Unsupervised Domain Adaptation

Contact Info

Product

Resources

About