A stochastic approximation approach to fixed instance selection

Yeo, Guo Feng Anders; Akman, David; Hudson, Irene L.; Chan, Jeffrey

doi:10.1016/j.ins.2023.01.090

Cited by 3 publications

(5 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Border instances are identified by analyzing the similarity index and sample labels, and are defined as instances with heterogeneous neighbors that are closest to samples from another class. The proposed algorithm in [27] selects the instances that contribute most to the performance of the machine learning model. The proposed method starts with a random subset of instances and trains a machine learning model on the selected subset.…”

Section: Model-based Samplingmentioning

confidence: 99%

Preserving Data Distribution in Sampling and Instance Selection with Renyi's Divergence

Sadoghi-Yazdi,

Ashkezari-Toussi,

Ramezanzadeh-Yazdi

2023

Preprint

View full text Add to dashboard Cite

This paper presents a novel method for sampling based on a distribution function. By leveraging Renyi's divergence criterion, a recursive formulation is directly derived from the disparity between the original and estimated distributions. The Gradient descent algorithm is employed to achieve this recursive equation, and then new samples are derived. Since this method uses the distribution of the original data, it has the ability to preserve the data distribution in the sampled dataset. In instances where the original distribution is unknown, Kernel Density Estimation is employed for estimation. Experimental results demonstrate that the proposed method effectively preserves the data distribution and prevents any alteration in the data concept while maintaining the predictability of the model over selected instances. More precisely, this method has successfully reduced the dataset size by approximately $70\%$, while the accuracy of the learning algorithm has remained unaffected.

show abstract

Section: Model-based Samplingmentioning

confidence: 99%

Preserving Data Distribution in Sampling and Instance Selection with Renyi's Divergence

Sadoghi-Yazdi,

Ashkezari-Toussi,

Ramezanzadeh-Yazdi

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Whereas the gain sequence is defined by an approximation of the Hessian matrix utilising the Barzilai and Borwein method (Barzilai & Borwein 1988). For more details refer to Yeo et al (2023).…”

Section: Fixed Instance Selectionmentioning

confidence: 99%

“…Whereas the second and potentially more serious issue is sub-optimal models, which become present in the form of over-fitting, fitting noisy entries or redundant features. Reduced model performance occurs when the dataset is excessively large in both the feature space (Kohavi & John 1997) and instance space (Yeo et al 2023). Dimensionality reduction is a common method for mitigating the aforementioned issues.…”

Section: Introductionmentioning

confidence: 99%

“…Recently we proposed SpFixedIS (Yeo et al 2023), a fixed instance selection algorithm which allowed for a user specified number of instances to select. Through a stochastic optimisation framework, SpFixedIS attempts to select a subset of instances which optimise a given performance with regards to a particular machine learning model.…”

Section: Introductionmentioning

confidence: 99%

“…SpFixedIS is a fixed wrapper instance selection algorithm (Yeo et al 2023), wherein the number of instances to select are user defined. The algorithm attempts to find the most representative instances of set cardinality with respect to a machine learning model and corresponding performance metric.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Visualising instance selection for improved explainability using feature extraction

2023

MODSIM2023, 25th International Congress on Modelling and Simulation.

View full text Add to dashboard Cite

Advancement in the collection and storage of data, alongside the modern emphasis of automated decision making have lead to datasets growing exponentially in size and complexity over the last four decades. Lever-aging excessively large data through traditional machine learning can lead to exorbitant run times, storage and general computational bloat, with the trained model potentially being sub-optimal (Kohavi & John 1997). Dimensionality reduction through selection and extraction are common methods of mitigating these issues. Extraction methods map the existing data to lower dimensional space whilst attempting to maintain the characteristics of the original dataset, whilst selection methods attempt to take a representative subset of the data. Alongside elevating the technical computational bloat, data reduction provides a parsimonious representation of the dataset, resulting in comparably simpler models which are more intrinsically interpretable. Therefore, data reduction techniques are included within Explainable Artificial Intelligence (XAI) (Barredo Arrieta et al. 2020). With the increasing reliance on automated decision making, the number of publications related to XAI has increased rapidly over the last ten years. A machine learning model should be not only accurate, but also transparent with an interpretable logic for proposed predictions. Therefore, machine learning models are used for both predictive purposes and retrospective data exploration and analysis.

show abstract