Distance-based k-nearest neighbors outlier detection method in large-scale traffic data

Dang, Taurus T.; Ngan, Henry Y. T.; Liu, Wei

doi:10.1109/icdsp.2015.7251924

Cited by 72 publications

(54 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Los autores presentan una discusión relacionada con los valores atípicos, los métodos utilizados para agrupar el conjunto de datos y, concluyen con que el algoritmo k-mean es el más popular en la agrupación de un conjunto de datos. Además, en otros estudios (Dang et al, 2015;Ganji, 2012;Gu et al, 2017;Malini y Pushpa, 2017;Mandhare y Idate, 2017;Sumaiya Thaseen y Aswani Kumar, 2017;Yan et al, 2016) se utilizan técnicas de minería de datos, métodos estadísticos o ambos. Para la detección de valores atípicos, comúnmente se han aplicado las técnicas del vecino más cercano (KNN) junto con otras para encontrar patrones inusuales durante el comportamiento de los datos o para mejorar el rendimiento del proceso.…”

Section: Trabajos Relacionadosunclassified

“…En otros trabajos se analizan las anomalías considerando varias columnas de la tupla, mas no la detección a nivel de cada columna (Dang, Ngan y Liu, 2015;Ganji, 2012;Gu et al, 2017;Malini y Pushpa, 2017;Mandhare y Idate, 2017;Sumaiya Thaseen y Aswani Kumar, 2017;Yan, You, Ji, Yin y Yang, 2016). Kuna et al (2013) presenta un procedimiento que utiliza árboles de decisión basados en el algoritmo C4.5, el cual aplica valores continuos y separa los posibles resultados en dos ramas.…”

unclassified

See 1 more Smart Citation

Detección de valores atípicos con técnicas de minería de datos y métodos estadísticos

Orellana

Cedillo

2020

Enfoque UTE

View full text Add to dashboard Cite

La detección de valores atípicos en el campo de la minería de datos (DM) y el descubrimiento de conocimiento a partir de datos (KDD) es de gran interés en áreas que requieren sistemas de soporte a la toma de decisiones, como, por ejemplo, en el área financiera, en donde mediante DM se pueden detectar fraudes financieros o encontrar errores producidos por los usuarios. Entonces, es esencial, evaluar la veracidad de la información, a través de métodos de detección de comportamientos inusuales en los datos. Este artículo propone un método para detectar valores que se consideran valores atípicos en una base de datos de datos de tipo nominal. El método implementa un algoritmo global de “k” vecinos más cercanos, un algoritmo de agrupamiento denominado k-means y un método estadístico denominado chi-cuadrado. La aplicación de estas técnicas ha sido implementada sobre una base de datos de clientes que han solicitado un crédito financiero. El experimento se realizó sobre un conjunto de datos con 1180 tuplas, en donde, deliberadamente se introdujeron valores atípicos. Los resultados demostraron que el método propuesto es capaz de detectar todos los valores atípicos introducidos.

show abstract

Section: Trabajos Relacionadosunclassified

unclassified

Detección de valores atípicos con técnicas de minería de datos y métodos estadísticos

Orellana

Cedillo

2020

Enfoque UTE

View full text Add to dashboard Cite

show abstract

“…We apply 'SAME'padding in all convolutional layers and use the hyperbolic tangent (tanh) as activation function g throughout the entire network, which is a common choice in feedforward neural networks. [1] Vice versa, when placing a fully connected layer after a convolutional layer, the inverse reindexing is performed. When using a convolutional layer as the first layer of an artificial neural network and the input is in fact a segment of a multivariate time series {Xi,t} i<M,t with M = M (0) features, no reindexing is required.…”

Section: Convolutional Neural Networkmentioning

confidence: 99%

“…Traditionally, anomaly detection is performed in the form of outlier detection in mathematical statistics. Numerous methods have been proposed, including but not limited to distance-and density-based techniques [1,2] and subspace-or submanifold-based techniques [3,4,5]. Most of these approaches make no explicit use of the concept of time and are therefore usually less suited for the analysis of time series.…”

Section: Introductionmentioning

confidence: 99%

“…Most of these approaches make no explicit use of the concept of time and are therefore usually less suited for the analysis of time series. * Correspondence: Lia.Ahrens@dfki.de 1 Deutsches Forschungszentrum für Künstliche Intelligenz, Trippstadter Straße 112 67663 Kaiserslautern, Germany Full list of author information is available at the end of the article Methods making explicit use of the temporal structure include classical models from statistical time series analysis such as autoregressive-moving average (ARMA) models [6], Kalman filters [7] or more general hidden Markov models [8], and rolling-window distance-based methods such as matrix profiles [9]. Distance analysing methods are effective for clean data but not robust against noise, whereas distributionbased methods from mathematical statistics are still powerful in the presence of noise, requiring dataspecific parametrisation.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A machine-learning phase classification scheme for anomaly detection in signals with periodic characteristics

Ahrens

Schotten

2019

EURASIP J. Adv. Signal Process.

View full text Add to dashboard Cite

In this paper we propose a novel machine-learning method for anomaly detection applicable to data with periodic characteristics where randomly varying period lengths are explicitly allowed. A multi-dimensional time series analysis is conducted by training a data-adapted classifier consisting of deep convolutional neural networks performing phase classification. The entire algorithm including data pre-processing, period detection, segmentation, and even dynamic adjustment of the neural networks is implemented for fully automatic execution. The proposed method is evaluated on three example datasets from the areas of cardiology, intrusion detection, and signal processing, presenting reasonable performance.

show abstract

Novel traffic congestion detection algorithms for smart city applications

Bawaneh

Simon

2022

Concurrency and Computation

View full text Add to dashboard Cite

Traffic congestion detection (TCD) techniques are becoming a critical component of traffic management systems. They can be considered a pre-step to address traffic jam problems, providing useful input for traffic management systems to predict and avoid traffic congestions' unwanted effects. In this research article, two novel real-time congestion detection algorithms are proposed, the TCD algorithm and the ensemble based traffic congestion detection (EB-TCD) algorithm. TCD can detect the congestion based on one traffic feature (i.e., speed, occupancy, or flow). TCD first cleans and preprocesses the traffic data. After, it computes the absolute of the first derivative of each sample in the traffic feature to determine its anomaly score. Then, the anomaly likelihood of each sample is computed to classify it as an anomaly or normal sample. On the other hand, EB-TCD utilizes the information contained in multiple traffic features in parallel by proposing an ensemble technique to combine the anomaly scores coming out from the different traffic features in one unified score. EB-TCD computes an anomaly score based on each traffic feature separately. Then, it votes for each sample in each traffic feature as an anomaly or not (from the corresponding traffic feature's point of view). After, the weight for each traffic feature is determined by studying the dissimilarity between their anomaly scores. These weights show how much each traffic feature explains the traffic behavior. Finally, the results are combined to form a unified anomaly score for each sample. The proposed statistical-based techniques are able to detect changes in traffic patterns while mitigating the effect of noise in traffic data. Moreover, the proposed algorithms' parameters were tuned and tailored for fast detection, which is a crucial feature for traffic management systems since fast detection accelerates decision-making. The computational complexity analysis has shown the simplicity of the proposed algorithms. Furthermore, the evaluation results have shown that the two algorithms outperform other widespread methods from the literature in terms of false alarm rate and detection time while keeping the detection rate at the same high level.

show abstract

Distance-based k-nearest neighbors outlier detection method in large-scale traffic data

Cited by 72 publications

References 10 publications

Detección de valores atípicos con técnicas de minería de datos y métodos estadísticos

Detección de valores atípicos con técnicas de minería de datos y métodos estadísticos

A machine-learning phase classification scheme for anomaly detection in signals with periodic characteristics

Novel traffic congestion detection algorithms for smart city applications

Contact Info

Product

Resources

About