Estimation of missing values in astronomical survey data: An improved local approach using cluster directed neighbor selection

Keerin, Phimmarin; Boongoen, Tossapon

doi:10.1016/j.ipm.2022.102881

Cited by 11 publications

(5 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, an inter-class overlap may dampen the quality of this local approach, as compared to the clustering-oriented technique such as SingleClus. The same problem is also witnessed for the task of imputing missing values, where clustering information can be exploited to improve the accuracy of estimates of those missing ones [37,38]. Nonetheless, the use of a single clustering seen with SingleClus may overlook patterns exhibited in data under examination.…”

Section: Resultsmentioning

confidence: 96%

Strengthening intrusion detection system for adversarial attacks: improved handling of imbalance classification problem

Pimsarn

Boongoen

Iam-On

et al. 2022

Complex Intell. Syst.

Self Cite

View full text Add to dashboard Cite

Most defence mechanisms such as a network-based intrusion detection system (NIDS) are often sub-optimal for the detection of an unseen malicious pattern. In response, a number of studies attempt to empower a machine-learning-based NIDS to improve the ability to recognize adversarial attacks. Along this line of research, the present work focuses on non-payload connections at the TCP stack level, which is generalized and applicable to different network applications. As a compliment to the recently published investigation that searches for the most informative feature space for classifying obfuscated connections, the problem of class imbalance is examined herein. In particular, a multiple-clustering-based undersampling framework is proposed to determine the set of cluster centroids that best represent the majority class, whose size is reduced to be on par with that of the minority. Initially, a pool of centroids is created using the concept of ensemble clustering that aims to obtain a collection of accurate and diverse clusterings. From that, the final set of representatives is selected from this pool. Three different objective functions are formed for this optimization driven process, thus leading to three variants of FF-Majority, FF-Minority and FF-Overall. Based on the thorough evaluation of a published dataset, four classification models and different settings, these new methods often exhibit better predictive performance than its baseline, the single-clustering undersampling counterpart and state-of-the-art techniques. Parameter analysis and implication for analyzing an extreme case are also provided as a guideline for future applications.

show abstract

Section: Resultsmentioning

confidence: 96%

Strengthening intrusion detection system for adversarial attacks: improved handling of imbalance classification problem

Pimsarn

Boongoen

Iam-On

et al. 2022

Complex Intell. Syst.

Self Cite

View full text Add to dashboard Cite

show abstract

“…One of which is the conventional thread that needs a data transformation process to extract discriminative features or attributes from images in a training dataset. See (Keerin & Boongoen, 2022) for examples of physics-based features used to deliver the profile of each image representing a new bright source. Then, they are extensively used to develop a conventional classifier such as a random forest or a single decision tree, artificial neural networks, k-nearest neighbours, and support vector machines, to name just a few (Sarker, 2021).…”

Section: Related Workmentioning

confidence: 99%

“…In past years, several studies (Tabacolde et al, 2018a,b) have designed this quest as a binary classification problem, to which conventional machine learning models can be applied. The main difficulty to obtain an accurate prediction model is the curse of imbalance class, where the minority class can contribute as tiny as 0.5% of the whole data collection (Iam-On et al, 2016;Haixianga et al, 2017;Keerin & Boongoen, 2022). A small improvement is made through balancing cardinality of samples belonging to both classes, using either an oversampling like SMOTE (Chawla et al, 2002) or a clustering-based undersampling counterpart (Lin et al, 2018a).…”

Section: Introductionmentioning

confidence: 99%

Improved detection of transient events in wide area sky survey using convolutional neural networks

Liu¹,

Boongoen²,

Iam-On³

2023

Data and Information Management

Self Cite

View full text Add to dashboard Cite

“…Imputation entails using statistical or machine learning procedures to estimate missing values in a data set. While still not widely accepted in the proteomics community, imputation has been standard practice for decades for analysis of gene expression [3] and clinical and epidemiological data [4], and more recently astronomy [5,6] and single-cell transcriptomic data [7,8]. Imputation methods for proteomics data (Table 1) fall into three broad categories: "singlevalue replacement" methods, in which all missing values are filled in with a single replacement value; "local similarity" methods, which use statistical models to learn patterns of local similarity in the data, for example between subsets of similar peptides or runs; and "global similarity" methods, which learn broad patterns of similarity across all peptides and runs.…”

Section: Introductionmentioning

confidence: 99%

Evaluating proteomics imputation methods with improved criteria

Harris

Fondrie²,

et al. 2023

Preprint

View full text Add to dashboard Cite

Quantitative measurements produced by tandem mass spectrometry proteomics experiments typically contain a large proportion of missing values. This missingness hinders reproducibility, reduces statistical power, and makes it difficult to compare across samples or experiments. Although many methods exist for imputing missing values in proteomics data, in practice, the most commonly used methods are among the worst performing. Furthermore, previous benchmarking studies have focused on relatively simple measurements of error, such as the mean-squared error between the imputed and the held-out observed values. Here we evaluate the performance of a set of commonly used imputation methods using three practical, "downstream-centric" criteria, which measure the ability of imputation methods to reconstruct differentially expressed peptides, identify new quantitative peptides, and improve peptide lower limit of quantification. Our evaluation spans several experiment types and acquisition strategies, including data-dependent and data-independent acquisition. We find that imputation does not necessarily improve the ability to identify differentially expressed peptides, but that it can identify new quantitative peptides and improve peptide lower limit of quantification. We find that MissForest is generally the best performing method per our downstream-centric criteria. We also argue that exisiting imputation methods do not properly account for the variance of peptide quantifications and highlight the need for methods that do.

show abstract

Estimation of missing values in astronomical survey data: An improved local approach using cluster directed neighbor selection

Cited by 11 publications

References 44 publications

Strengthening intrusion detection system for adversarial attacks: improved handling of imbalance classification problem

Strengthening intrusion detection system for adversarial attacks: improved handling of imbalance classification problem

Improved detection of transient events in wide area sky survey using convolutional neural networks

Evaluating proteomics imputation methods with improved criteria

Contact Info

Product

Resources

About