Submodular Observation Selection and Information Gathering for Quadratic Models

Hashemi, Abolfazl; Ghasemi, Mahsa; Vikalo, Haris; Topcu, Ufuk

doi:10.48550/arxiv.1905.09919

Cited by 5 publications

(16 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, unlike us, these setups do not consider any validation constraint. Our work is also connected with batch active learning methods [42,15,29,39], that aim to select examples from training data in order to minimize the labeling cost. In contrast, our setup has access to all the labels and it aims to select data to improve efficiency.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Training Data Subset Selection for Regression with Controlled Generalization Error

Sivasubramanian,

Iyer,

Ramakrishnan

et al. 2021

Preprint

View full text Add to dashboard Cite

Data subset selection from a large number of training instances has been a successful approach toward efficient and cost-effective machine learning. However, models trained on a smaller subset may show poor generalization ability. In this paper, our goal is to design an algorithm for selecting a subset of the training data, so that the model can be trained quickly, without significantly sacrificing on accuracy. More specifically, we focus on data subset selection for L2 regularized regression problems and provide a novel problem formulation which seeks to minimize the training loss with respect to both the trainable parameters and the subset of training data, subject to error bounds on the validation set. We tackle this problem using several technical innovations. First, we represent this problem with simplified constraints using the dual of the original training problem and show that the objective of this new representation is a monotone and α-submodular function, for a wide variety of modeling choices. Such properties lead us to develop SELCON, an efficient majorization-minimization algorithm for data subset selection, that admits an approximation guarantee even when the training provides an imperfect estimate of the trained model. Finally, our experiments on several datasets show that SELCON trades off accuracy and efficiency more effectively than the current state-of-the-art.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Subset selection for linear regression problems has been widely studied in literature [15,10]. Except for [10], these approaches optimize measures associated with the covariance matrix, rather than explicitly minimizing the training loss subject to the validation set error bound.…”

Section: α-Submodularity Of F (S)mentioning

confidence: 99%

Training Data Subset Selection for Regression with Controlled Generalization Error

Sivasubramanian,

Iyer,

Ramakrishnan

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…To solve this optimization problem, we present a greedy approach which, like an ordinary greedy submodular maximization algorithm, enjoys approximation bounds. However, since some of the optimization objectives we consider are only weakly submodular, they admit some special approximation bounds which have been proposed recently in [33].…”

Section: Proposed Approachmentioning

confidence: 99%

“…Finally we conclude this section with this note that our work closely resembles SLANT [18] and is built upon the modeling framework proposed by SLANT, but the major difference is that SLANT assumes the entire event stream as endogenous, whereas our work is motivated towards exploring various techniques for systematically demarcating the externalities from the heterogeneous event stream. Similarly, our proposed algorithms are closely influenced by recent progress in subset selection literature [33] where authors deal with designing new alphabetical optimality criteria for quadratic models. However, our work is motivated towards investing the subset selection problem for linear models in a temporal setting.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Demarcating Endogenous and Exogenous Opinion Dynamics: An Experimental Design Approach

Koley

Saha

Bhattacharya

et al. 2021

ACM Trans. Knowl. Discov. Data

View full text Add to dashboard Cite

The networked opinion diffusion in online social networks is often governed by the two genres of opinions— endogenous opinions that are driven by the influence of social contacts among users, and exogenous opinions which are formed by external effects like news and feeds. Accurate demarcation of endogenous and exogenous messages offers an important cue to opinion modeling, thereby enhancing its predictive performance. In this article, we design a suite of unsupervised classification methods based on experimental design approaches, in which, we aim to select the subsets of events which minimize different measures of mean estimation error. In more detail, we first show that these subset selection tasks are NP-Hard. Then we show that the associated objective functions are weakly submodular, which allows us to cast efficient approximation algorithms with guarantees. Finally, we validate the efficacy of our proposal on various real-world datasets crawled from Twitter as well as diverse synthetic datasets. Our experiments range from validating prediction performance on unsanitized and sanitized events to checking the effect of selecting optimal subsets of various sizes. Through various experiments, we have found that our method offers a significant improvement in accuracy in terms of opinion forecasting, against several competitors.

show abstract

Classification Under Human Assistance

Okati

Zarezade

et al. 2021

AAAI

View full text Add to dashboard Cite

Most supervised learning models are trained for full automation. However, their predictions are sometimes worse than those by human experts on some specific instances. Motivated by this empirical observation, our goal is to design classifiers that are optimized to operate under different automation levels. More specifically, we focus on convex margin-based classifiers and first show that the problem is NP-hard. Then, we further show that, for support vector machines, the corresponding objective function can be expressed as the difference of two functions f = g - c, where g is monotone, non-negative and gamma-weakly submodular, and c is non-negative and modular. This representation allows a recently introduced deterministic greedy algorithm, as well as a more efficient randomized variant of the algorithm, to enjoy approximation guarantees at solving the problem. Experiments on synthetic and real-world data from several applications in medical diagnosis illustrate our theoretical findings and demonstrate that, under human assistance, supervised learning models trained to operate under different automation levels can outperform those trained for full automation as well as humans operating alone.

show abstract

Submodular Observation Selection and Information Gathering for Quadratic Models

Cited by 5 publications

References 18 publications

Training Data Subset Selection for Regression with Controlled Generalization Error

Training Data Subset Selection for Regression with Controlled Generalization Error

Demarcating Endogenous and Exogenous Opinion Dynamics: An Experimental Design Approach

Classification Under Human Assistance

Contact Info

Product

Resources

About