A Robust UCB scheme for active learning in regression from strategic crowds

Abstract

We study the problem of training an accurate linear regression model by procuring labels from multiple noisy crowd annotators, under a budget constraint. We propose a Bayesian model for linear regression in crowdsourcing and use variational inference for parameter estimation. To minimize the number of labels crowdsourced from the annotators, we adopt an active learning approach. In this specific context, we prove the equivalence of well-studied criteria of active learning like entropy minimization and expected error reduction. Interestingly, we observe that we can decouple the problems of identifying an optimal unlabeled instance and identifying an annotator to label it. We observe a useful connection between the multi-armed bandit framework and the annotator selection in active learning. Due to the nature of the distribution of the rewards on the arms, we use the Robust Upper Confidence Bound (UCB) scheme with truncated empirical mean estimator to solve the annotator selection problem. This yields provable guarantees on the regret. We further apply our model to the scenario where annotators are strategic and design suitable incentives to induce them to put in their best efforts.

Corruption-tolerant bandit learning

“…Literature on bandits is too vast to be surveyed here. Starting with the early work of Auer et al (2002a) on multi-armed bandits (MAB), the field has seen progress in linear bandits (Abbasi-Yadkori et al 2011), contextual bandits (Chu et al 2011), as well as applications to recommendation (Li et al 2010), advertising (Chakrabarti et al 2008), web analytics (Tang et al 2013), crowdsourcing (Padmanabhan et al 2016), and mobile health (Tewari and Murphy 2017).…”

“…There has been recent interest in developing bandit algorithms where the arm responses are samples from heavy-tailed distributions such as the works of Bubeck et al (2013), Medina and Yang (2016), Padmanabhan et al (2016). A point of confusion may arise here since these algorithms are also sometimes referred to as "robust" algorithms.…”

“…The recent years have witnessed a surge in the applications of online learning, especially those of explore-exploit techniques such as multi-armed bandits and linear-contextual bandits, to online recommendation (Li et al 2010), online advertising (Chakrabarti et al 2008), web analytics (Tang et al 2013), crowdsourcing (Padmanabhan et al 2016), and even mobile health (Tewari and Murphy 2017). The result has been a diverse and rich literature, accompanied by a deep understanding of how these algorithms work on large-scale data.…”