Ordinal Label Proportions

Abstract

In Machine Learning, it is common to distinguish different degrees of supervision, ranging from fully supervised to completely unsupervised scenarios. However, lying in between those, the Learning from Label Proportions (LLP) setting [19] assumes the training data is provided in the form of bags, and the only supervision comes through the proportion of each class in each bag. In this paper, we present a novel version of the LLP paradigm where the relationship among the classes is ordinal. While this is a highly relevant scenario (e.g. customer surveys where the results can be divided into various degrees of satisfaction), it is as yet unexplored in the literature. We refer to this setting as Ordinal Label Proportions (OLP). We formally define the scenario and introduce an efficient algorithm to tackle it. We test our algorithm on synthetic and benchmark datasets. Additionally, we present a case study examining a dataset gathered from the Research Excellence Framework that assesses the quality of research in the United Kingdom.

Label Propagation for Learning With Label Proportions

“…This approach enjoys uniform convergence under the following (strong) assumption: conditioned on its label, a point is independent of its bag assignment i, namely, p(x|y, i) = p(x|y). Extensions of MeanMap can be found in [12,13].…”

Hypothesis Testing for Class-Conditional Noise Using Local Maximum Likelihood

“…In Poyiadzi et al (2022), the authors introduced a framework that follows from the asymptotic properties of Maximum Likelihood Estimation (MLE) of the logistic regression model and thus its correctness relies on whether the assumptions of the model are met. In practice, the linear assumptions behind the method mean that, unless there is an element of control over the data generating process, the blind application of the test can lead to suboptimal outcomes.…”

“…• We extend the parametric form of the hypothesis test for class-conditional label noise proposed in Poyiadzi et al (2022), and consider a nonparametric estimation of the underlying regression function based on local likelihood models. • We thoroughly compare the strengths and weaknesses of these two methods respectively, and provide guidelines for machine learning practitioners to know which one to use given a dataset.…”

“…Testing for Linearity In this work, we introduce an extension of the tests in Poyiadzi et al (2022) that replaces the use of a parametric model, with a nonparametric model. For a practitioner, it would be necessary to know whether a parametric (or linear) fit is suitable for their needs.…”

Active Learning with Label Proportions

“…Although this relaxes the assumption of infallibility of the oracle, we argue that exploring varying degrees of supervision can lead to an easier and simpler interaction in between the algorithm and the oracle. In this paper, we cast our problem as an instance of the Learning from Label Proportions (LLP) setting [8,9,10,11,12]. Figure 1 illustrates this idea.…”