Active learning via query synthesis and nearest neighbour search

Abstract

Active learning has received great interests from researchers due to its ability to reduce the amount of supervision required for effective learning. As the core component of active learning algorithms, query synthesis and pool-based sampling are two main scenarios of querying considered in the literature. Query synthesis features low querying time, but only has limited applications as the synthesized query might be unrecognizable to human oracle. As a result, most efforts have focused on pool-based sampling in recent years, although it is much more time-consuming. In this paper, we propose new strategies for a novel querying framework that combines query synthesis and pool-based sampling. It overcomes the limitation of query synthesis, and has the advantage of fast querying. The basic idea is to synthesize an instance close to the decision boundary using labeled data, and then select the real instance closest to the synthesized one as a query. For this purpose, we propose a synthesis strategy, which can synthesize instances close to the decision boundary and spreading along the decision boundary. Since the synthesis only depends on the relatively small labelled set, instead of evaluating the entire unlabeled set as many other active learning algorithms do, our method has the advantage of efficiency. In order to handle more complicated data and make our framework compatible with powerful kernel-based learners, we also extend our method to kernel version. Experiments on several real-world data sets show that our method has significant advantage on time complexity and similar performance compared to pool-based uncertainty sampling methods.

Visually-Enabled Active Deep Learning for (Geo) Text and Image Classification: A Review

“…Membership query synthesis was proposed in [45], and further developed and extended in [46][47][48][49][50]. In this scenario, the machine learner knows the definition of the instance space (e.g., feature dimensions and ranges are known).…”

“…The learner can generate (i.e., synthesize) a new instance (e.g., an image or a piece of text) from scratch (thus one that meets the parameters of the instance space, but may or may not actually exist [28]) that satisfies the instance space definition, and then enlist an annotator for labeling [49,50]. Query synthesis can synthesize a new artificial (membership) query from scratch using a small amount of labelled data-it is therefore very efficient [49]. Query synthesis is often tractable and efficient for finite problem domains [46].…”

Active Learning for Black-Box Adversarial Attacks in EEG-Based Brain-Computer Interfaces

“…Active learning is an effective way to reduce the data labeling effort, by actively selecting the most useful instances to label. There are two main scenarios of active learning in the literature: query synthesis [29]- [32] and sampling. The latter can be further divided into stream-based sampling [33], [34] and pool-based sampling [35]- [38].…”

“…Therefore, we synthesize the next query along the mid-perpendicular direction after we find an opposite-pair close enough to the decision boundary. Specifically, we find the opposite-pair's orthogonal vector by Gram-Schmidt process [32], set the magnitude of the orthogonal vector to q, then move it to a more precise midpoint. The details are shown in Algorithm 2.…”

‘Security Theater’: On the Vulnerability of Classifiers to Exploratory Attacks

“…Random sampling can lead to wasted probes, with no new information added. The query synthesis strategy of [24] generates samples close to the decision boundary and spreads these samples across the boundary, for better learning of the decision landscape. The approach in [24] was used for selecting samples for active learning.…”