Dynamic Ensemble Active Learning: A Non-Stationary Bandit with Expert Advice

Abstract

Active learning aims to reduce annotation cost by predicting which samples are useful for a human teacher to label. However it has become clear there is no best active learning algorithm. Inspired by various philosophies about what constitutes a good criteria, different algorithms perform well on different datasets. This has motivated research into ensembles of active learners that learn what constitutes a good criteria in a given scenario, typically via multi-armed bandit algorithms. Though algorithm ensembles can lead to better results, they overlook the fact that not only does algorithm efficacy vary across datasets, but also during a single active learning session. That is, the best criteria is non-stationary. This breaks existing algorithms' guarantees and hampers their performance in practice. In this paper, we propose dynamic ensemble active learning as a more general and promising research direction. We develop a dynamic ensemble active learner based on a non-stationary multi-armed bandit with expert advice algorithm. Our dynamic ensemble selects the right criteria at each step of active learning. It has theoretical guarantees, and shows encouraging results on 13 popular datasets.

Onception: Active Learning with Expert Advice for Real World Machine Translation

“…Their approach performed in line with the best two individual query strategies and outperformed Hsu and Lin (2015). Pang et al (2018) proposed a modification of multi-armed bandits with experts, to account for non-stationary loss functions (i.e., the best expert might vary over time), in binary and multi-class classification tasks. Their approach outperformed or performed in line with the best individual strategies and outperformed both Baram, El-Yaniv, and Luz (2004) and Hsu and Lin (2015) in non-stationary datasets.…”

“…Since our end goal is to improve the MT ensemble, we measure its improvement at each weight update by considering the expected regret R M for not choosing the best MT system's translation at each iteration, up to the current iteration T (Eq. 9), which can be seen as a dynamic regret (Pang et al 2018). Note that this regret formulation deviates from the traditional formulation, in that we compare the forecaster to the best sequence of decisions overall (whose cumulative loss is given by T t=1 min j=1,...,J j,t ), instead of the best expert overall (whose cumulative loss would be given by min j=1,...,J T t=1 j,t ).…”

A review and experimental analysis of active learning over crowdsourced data

“…Many other data-driven approaches for pool-based AL processes have been proposed recently. While Bachman et al (2017) and Pang et al (2018b) used RL to build the learning model, Liu et al (2018) formulated learning AL strategies as an imitation learning problem (i.e., the machine is trained to perform a task from demonstrations by learning a mapping between observations and actions), Contardo et al (2017) and Ravi and Larochelle (2018) Pang et al (2018a) extended the LSA approach using non-stationary multi-armed bandit with expert advice.…”

From Weakly Supervised Learning to Biquality Learning: an Introduction

“…Conversely, selecting an example in an already sampled region allows to locally refine the predictive model. We do not intend to provide an exhaustive overview of existing AL strategies and refer to [37], [38] for a detailed overview, [39]- [41] for some recent benchmark and a new way to treat uncertainty in [42] Another meta active learning paradigm exists, which combines conventional strategies using bandit algorithms [43]- [48]. These meta-learning algorithms intend to select online the best AL strategy according to the observed improvements of the classifier.…”