A sequential sampling strategy for adaptive classification of computationally expensive data

Singh, Prashant; Herten, Joachim van der; Deschrijver, Dirk; Couckuyt, Ivo; Dhaene, Tom

doi:10.1007/s00158-016-1584-1

Cited by 17 publications

(14 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To determine the informativeness of unlabeled data, such approaches use the model's predictive mean and variance to either model the uncertainty of unlabeled data [18,19,20,21,22,23] or estimate the expected model change affected by unlabeled data [24,25]. Instead of using a probabilistic model, others use deterministic models (e.g., support vector machines) and combine the classification results with some geometric rules for selecting queries [26,7,27,28]. Another body of work models active learning MD-17-1152 W. Chen, M. Fuge as a multi-armed bandit problem, automatically selecting queries to balance exploitation and exploration via a performance estimate [29,30].…”

Section: Sequential Sampling and Active Learningmentioning

confidence: 99%

Beyond the Known: Detecting Novel Feasible Domains Over an Unbounded Design Space

Chen

Fuge

2017

Journal of Mechanical Design

View full text Add to dashboard Cite

To solve a design problem, sometimes it is necessary to identify the feasible design space. For design spaces with implicit constraints, sampling methods are usually used. These methods typically bound the design space; that is, limit the range of design variables. But bounds that are too small will fail to cover all possible designs; while bounds that are too large will waste sampling budget. This paper tries to solve the problem of efficiently discovering (possibly disconnected) feasible domains in an unbounded design space. We propose a data-driven adaptive sampling technique-ε-margin sampling, which both learns the domain boundary of feasible designs, while also expanding our knowledge of the design space as available budget increases. This technique is dataefficient, in that it makes principled probabilistic trade-offs between refining existing domain boundaries versus expanding the design space.We demonstrate that this method can better identify feasible domains on standard test functions compared to both random and active sampling (via uncertainty sampling). However, a fundamental problem when applying adaptive sampling to real world designs is that designs often have high dimensionality and thus require (in the worst case) exponentially more samples per dimension. We show how coupling Design Manifolds with ε-margin * Corresponding author.sampling allows us to actively expand high-dimensional design spaces without incurring this exponential penalty.We demonstrate this on real-world examples of glassware and bottle design, where our method discovers designs that have different appearance and functionality from its initial design set.

show abstract

Section: Sequential Sampling and Active Learningmentioning

confidence: 99%

Beyond the Known: Detecting Novel Feasible Domains Over an Unbounded Design Space

Chen

Fuge

2017

Journal of Mechanical Design

View full text Add to dashboard Cite

show abstract

“…In the context of this work, we have used various adaptive sampling schemes that perform exploration and/or exploitation in the design space (NV, EDSD, PoF, and Entropy [18,[22][23][24]). These techniques are discussed in the following subsections.…”

Section: Classification Methodsmentioning

confidence: 99%

“…Data-efficient machine learning (DEML) techniques have proven useful at reducing the computational requirements of a variety of engineering problems [18][19][20][21]. These techniques can be applied to micromagnetic problems for various objectives.…”

Section: Introductionmentioning

confidence: 99%

“…Several state-of-the-art sampling strategies are combined with machine learning classification models. This paper evaluates the performance of the Explicit Design Space Decomposition (EDSD), Neighborhood-Voronoi (NV), Probability of Feasibility (PoF), and Entropy [18,[22][23][24] sampling strategies. The performance of each technique is compared by three classifiers: Support Vector Machines (SVM), Gaussian process (GP), and Logistic regression (LR) [25][26][27] built on a training data that is obtained by adaptive sampling strategies.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Fast Characterization of Input-Output Behavior of Non-Charge-Based Logic Devices by Machine Learning

et al. 2020

Self Cite

View full text Add to dashboard Cite

Non-charge-based logic devices are promising candidates for the replacement of conventional complementary metal-oxide semiconductors (CMOS) devices. These devices utilize magnetic properties to store or process information making them power efficient. Traditionally, to fully characterize the input-output behavior of these devices a large number of micromagnetic simulations are required, which makes the process computationally expensive. Machine learning techniques have been shown to dramatically decrease the computational requirements of many complex problems. We use state-of-the-art data-efficient machine learning techniques to expedite the characterization of their behavior. Several intelligent sampling strategies are combined with machine learning (binary and multi-class) classification models. These techniques are applied to a magnetic logic device that utilizes direct exchange interaction between two distinct regions containing a bistable canted magnetization configuration. Three classifiers were developed with various adaptive sampling techniques in order to capture the input-output behavior of this device. By adopting an adaptive sampling strategy, it is shown that prediction accuracy can approach that of full grid sampling while using only a small training set of micromagnetic simulations. Comparing model predictions to a grid-based approach on two separate cases, the best performing machine learning model accurately predicts 99.92% of the dense test grid while utilizing only 2.36% of the training data respectively.

show abstract

“…We evaluate the performance of AES in capturing feasible domains using both synthesized and real-world examples. The performance is measured by the F1 score, which is expressed as We compare AES with two conventional bounded adaptive sampling methods -the Neighborhood-Voronoi (NV) algorithm (Singh et al, 2017) and the straddle heuristic (Bryan et al, 2006). We also investigate the effects of noise and dimensionality on AES.…”

Section: Experimental Evaluationmentioning

confidence: 99%

Active expansion sampling for learning feasible domains in an unbounded input space

Chen

Fuge

2018

Struct Multidisc Optim

View full text Add to dashboard Cite

Many engineering problems require identifying feasible domains under implicit constraints. One example is finding acceptable car body styling designs based on constraints like aesthetics and functionality. Current active-learning based methods learn feasible domains for bounded input spaces. However, we usually lack prior knowledge about how to set those input variable bounds. Bounds that are too small will fail to cover all feasible domains; while bounds that are too large will waste query budget. To avoid this problem, we introduce Active Expansion Sampling (AES), a method that identifies (possibly disconnected) feasible domains over an unbounded input space. AES progressively expands our knowledge of the input space, and uses successive exploitation and exploration stages to switch between learning the decision boundary and searching for new feasible domains. We show that AES has a misclassification loss guarantee within the explored region, independent of the number of iterations or labeled samples. Thus it can be used for real-time prediction of samples' feasibility within the explored region. We evaluate AES on three test examples and compare AES with two adaptive sampling methods -the Neighborhood-Voronoi algorithm and the straddle heuristic -that operate over fixed input variable bounds.

show abstract

A sequential sampling strategy for adaptive classification of computationally expensive data

Cited by 17 publications

References 41 publications

Beyond the Known: Detecting Novel Feasible Domains Over an Unbounded Design Space

Beyond the Known: Detecting Novel Feasible Domains Over an Unbounded Design Space

Fast Characterization of Input-Output Behavior of Non-Charge-Based Logic Devices by Machine Learning

Active expansion sampling for learning feasible domains in an unbounded input space

Contact Info

Product

Resources

About