A New Algorithm for Detecting Outliers in Linear Regression

Satman, Mehmet Hakan

doi:10.5539/ijsp.v2n3p101

Cited by 21 publications

(6 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the paper [ 42 ] such an approach was used to detect anomalies in the phasor measurement units (PMU) of multivariate streaming. On the other hand, in the paper [ 43 ] a linear regression was used to detect anomalies. Furthermore, kernel density estimation-based methods were used as outlier detection approaches [ 44 ].…”

Section: Related Workmentioning

confidence: 99%

Anomaly Detection Module for Network Traffic Monitoring in Public Institutions

Wawrowski

Bialas

Kajzer

et al. 2023

Sensors

View full text Add to dashboard Cite

It seems to be a truism to say that we should pay more and more attention to network traffic safety. Such a goal may be achieved with many different approaches. In this paper, we put our attention on the increase in network traffic safety based on the continuous monitoring of network traffic statistics and detecting possible anomalies in the network traffic description. The developed solution, called the anomaly detection module, is mostly dedicated to public institutions as the additional component of the network security services. Despite the use of well-known anomaly detection methods, the novelty of the module is based on providing an exhaustive strategy of selecting the best combination of models as well as tuning the models in a much faster offline mode. It is worth emphasizing that combined models were able to achieve 100% balanced accuracy level of specific attack detection.

show abstract

Section: Related Workmentioning

confidence: 99%

Anomaly Detection Module for Network Traffic Monitoring in Public Institutions

Wawrowski

Bialas

Kajzer

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…The earliest studies on outlier detection date back to the 1960s, when researchers considered outliers to be noise and not to contain any valuable information. However, “one person's noise may be another person's signal” 12 , in-depth analysis of outliers can reveal information of significant value hidden in the data 13 .…”

Section: Related Workmentioning

confidence: 99%

Fluctuation-based outlier detection

Zuo

Chu

et al. 2023

Sci Rep

View full text Add to dashboard Cite

Outlier detection is an important topic in machine learning and has been used in a wide range of applications. Outliers are objects that are few in number and deviate from the majority of objects. As a result of these two properties, we show that outliers are susceptible to a mechanism called fluctuation. This article proposes a method called fluctuation-based outlier detection (FBOD) that achieves a low linear time complexity and detects outliers purely based on the concept of fluctuation without employing any distance, density or isolation measure. Fundamentally different from all existing methods. FBOD first converts the Euclidean structure datasets into graphs by using random links, then propagates the feature value according to the connection of the graph. Finally, by comparing the difference between the fluctuation of an object and its neighbors, FBOD determines the object with a larger difference as an outlier. The results of experiments comparing FBOD with eight state-of-the-art algorithms on eight real-worlds tabular datasets and three video datasets show that FBOD outperforms its competitors in the majority of cases and that FBOD has only 5% of the execution time of the fastest algorithm. The experiment codes are available at: https://github.com/FluctuationOD/Fluctuation-based-Outlier-Detection.

show abstract

“…The key difference is that parametric methods assume that the data come from a parametric distribution, so that fitting such a distribution amounts to learning the parameters of the assumed parametric distribution. Common parametric methods for OD include using Gaussian mixture models (GMM) [41] and linear regression [42]. In contrast, nonparametric methods do not assume a parametric model for the data.…”

Section: Statistical Modelsmentioning

confidence: 99%

ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functions

Li,

Zhao,

et al. 2022

Preprint

View full text Add to dashboard Cite

Outlier detection refers to the identification of data points that deviate from a general data distribution. Existing unsupervised approaches often suffer from high computational cost, complex hyperparameter tuning, and limited interpretability, especially when working with large, high-dimensional datasets. To address these issues, we present a simple yet effective algorithm called ECOD (Empirical-Cumulative-distribution-based Outlier Detection), which is inspired by the fact that outliers are often the "rare events" that appear in the tails of a distribution. In a nutshell, ECOD first estimates the underlying distribution of the input data in a nonparametric fashion by computing the empirical cumulative distribution per dimension of the data. ECOD then uses these empirical distributions to estimate tail probabilities per dimension for each data point. Finally, ECOD computes an outlier score of each data point by aggregating estimated tail probabilities across dimensions. Our contributions are as follows: (1) we propose a novel outlier detection method called ECOD, which is both parameter-free and easy to interpret; (2) we perform extensive experiments on 30 benchmark datasets, where we find that ECOD outperforms 11 state-of-the-art baselines in terms of accuracy, efficiency, and scalability; and (3) we release an easy-to-use and scalable (with distributed support) Python implementation for accessibility and reproducibility.

show abstract

A New Algorithm for Detecting Outliers in Linear Regression

Cited by 21 publications

References 37 publications

Anomaly Detection Module for Network Traffic Monitoring in Public Institutions

Anomaly Detection Module for Network Traffic Monitoring in Public Institutions

Fluctuation-based outlier detection

ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functions

Contact Info

Product

Resources

About