Detection of outliers in high-dimensional data using <i>nu</i>-support vector regression

Rashid, Abdullah Mohammed; Midi, Habshah; Dhhan, Waleed; Arasan, Jayanthi

doi:10.1080/02664763.2021.1911965

Cited by 5 publications

(3 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These data points are embedded vectors of climate events. Since there are 1–10% anomalies in a dataset 14 , we conduct the experiments to validate the performance of the proposed model by selecting 1–10% anomalies. For example, we assume that 10% of data points are selected as outliers in each dataset.…”

Section: Resultsmentioning

confidence: 99%

Entropy-based dynamic graph embedding for anomaly detection on multiple climate time series

Jung

2021

Sci Rep

View full text Add to dashboard Cite

Abnormal climate event is that some meteorological conditions are extreme in a certain time interval. The existing methods for detecting abnormal climate events utilize supervised learning models to learn the abnormal patterns, but they cannot detect the untrained patterns. To overcome this problem, we construct a dynamic graph by discovering the correlation among the climate time series and propose a novel dynamic graph embedding model based on graph entropy called EDynGE to discriminate anomalies. The graph entropy measurement quantifies the information of the graphs and constructs the embedding space. We conducted experiments on synthetic datasets and real-world meteorological datasets. The results showed that EdynGE model achieved a better F1-score than the baselines by 43.2%, and the number of days of abnormal climate events has increased by 304.5 days in the past 30 years.

show abstract

Section: Resultsmentioning

confidence: 99%

Entropy-based dynamic graph embedding for anomaly detection on multiple climate time series

Jung

2021

Sci Rep

View full text Add to dashboard Cite

show abstract

“…A dataset may contain multiple outliers, posing challenges in detecting and addressing the masking and swamping effects [7]. Various methods have been employed for multiple outlier detection, including the fully efficient one-step procedure (GY) proposed by Gervini and Yohai (2002) [8], the least trimmed squares (LTS) [9], and the MM-estimators [10].…”

Section: Introductionmentioning

confidence: 99%

Outlier detection in spatial error models using modified thresholding-based iterative procedure for outlier detection approach

Cai,

Hu,

Yang

et al. 2024

BMC Med Res Methodol

View full text Add to dashboard Cite

Background Outliers, data points that significantly deviate from the norm, can have a substantial impact on statistical inference and provide valuable insights in data analysis. Multiple methods have been developed for outlier detection, however, almost all available approaches fail to consider the spatial dependence and heterogeneity in spatial data. Spatial data has diverse formats and semantics, requiring specialized outlier detection methodology to handle these unique properties. For now, there is limited research exists on robust spatial outlier detection methods designed specifically under the spatial error model (SEM) structure. Method We propose the Spatial-Θ-Iterative Procedure for Outlier Detection (Spatial-Θ-IPOD), which utilizes a mean-shift vector to identify outliers within the SEM. Our method enables an effective detection of spatial outliers while also providing robust coefficient estimates. To assess the performance of our approach, we conducted extensive simulations and applied it to a real-world empirical study using life expectancy data from multiple countries. Results Simulation results showed that the masking and JD (Joint Detection) indicators of our Spatial-Θ-IPOD method outperformed several commonly used methods, even in high-dimensional scenarios, demonstrating stable performance. Conversely, the Θ-IPOD method proved to be ineffective in detecting outliers when spatial correlation was present. Moreover, our model successfully provided reliable coefficient estimation alongside outlier detection. The proposed method consistently outperformed other models (both robust and non-robust) in most cases. In the empirical study, our proposed model successfully detected outliers and provided valuable insights in the modeling process. Conclusions Our proposed Spatial-Θ-IPOD offers an effective solution for detecting spatial outliers for SEM while providing robust coefficient estimates. Notably, our approach showcases its relative superiority even in the presence of high leverage points. By successfully identifying outliers, our method enhances the overall understanding of the data and provides valuable insights for further analysis.

show abstract

“…Support Vector Regression (SVR), effective in high-dimensional spaces and robust to outliers, demands careful selection of kernel and parameters due to its computational intensity [ 25 ]. Decision Tree Regression, with its capability to handle non-linearity and interactions, is visually interpretable but prone to overfitting and sensitive to small variations in data.…”

Section: Introductionmentioning

confidence: 99%

Estimation and Prediction of the Polymers’ Physical Characteristics Using the Machine Learning Models

Malashin,

Tynchenko,

Nelyub

et al. 2023

Polymers

View full text Add to dashboard Cite

This article investigates the utility of machine learning (ML) methods for predicting and analyzing the diverse physical characteristics of polymers. Leveraging a rich dataset of polymers’ characteristics, the study encompasses an extensive range of polymer properties, spanning compressive and tensile strength to thermal and electrical behaviors. Using various regression methods like Ensemble, Tree-based, Regularization, and Distance-based, the research undergoes thorough evaluation using the most common quality metrics. As a result of a series of experimental studies on the selection of effective model parameters, those that provide a high-quality solution to the stated problem were found. The best results were achieved by Random Forest with the highest R2 scores of 0.71, 0.73, and 0.88 for glass transition, thermal decomposition, and melting temperatures, respectively. The outcomes are intricately compared, providing valuable insights into the efficiency of distinct ML approaches in predicting polymer properties. Unknown values for each characteristic were predicted, and a method validation was performed by training on the predicted values, comparing the results with the specified variance values of each characteristic. The research not only advances our comprehension of polymer physics but also contributes to informed model selection and optimization for materials science applications.

show abstract

Detection of outliers in high-dimensional data using nu-support vector regression

Cited by 5 publications

References 29 publications

Entropy-based dynamic graph embedding for anomaly detection on multiple climate time series

Entropy-based dynamic graph embedding for anomaly detection on multiple climate time series

Outlier detection in spatial error models using modified thresholding-based iterative procedure for outlier detection approach

Estimation and Prediction of the Polymers’ Physical Characteristics Using the Machine Learning Models

Contact Info

Product

Resources

About