Autonomous profile-based anomaly detection system using principal component analysis and flow analysis

Fernandes, Gilberto; Rodrigues, Joel J. P. C.; Proença, Mário Lemes

doi:10.1016/j.asoc.2015.05.019

Cited by 43 publications

(23 citation statements)

References 35 publications

(39 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, the response information can be symbolised by a reduced fixed set of dimensions without much loss of information (Hamamoto et al, 2017). Besides, It analyses noteworthy information data from logs to finds the important activity time intervals among the data set, and afterward diminish them, so this new set can professionally characterize to the consistent conduct of a network segment (Fernandes, Rodrigues, & Proença, 2015).…”

Section: Random Forest (Rf)mentioning

confidence: 99%

Real-time big data processing for anomaly detection: A Survey

Habeeb¹,

Nasaruddin²,

Gani³

et al. 2019

International Journal of Information Management

273

View full text Add to dashboard Cite

The advent of connected devices and omnipresence of Internet have paved way for intruders to attack networks, which leads to cyber-attack, financial loss, information theft in healthcare, and cyber war. Hence, network security analytics has become an important area of concern and has gained intensive attention among researchers, off late, specifically in the domain of anomaly detection in network, which is considered crucial for network security. However, preliminary investigations have revealed that the existing approaches to detect anomalies in network are not effective enough, particularly to detect them in real time. The reason for the inefficacy of current approaches is mainly due the amassment of massive volumes of data though the connected devices. Therefore, it is crucial to propose a framework that effectively handles real time big data processing and detect anomalies in networks. In this regard, this paper attempts to address the issue of detecting anomalies in real time. Respectively, this paper has surveyed the state-of-the-art real-time big data processing technologies related to anomaly detection and the vital characteristics of associated machine learning algorithms. This paper begins with the explanation of essential contexts and taxonomy of real-time big data processing, anomalous detection, and machine learning algorithms, followed by the review of big data processing technologies. Finally, the identified research challenges of real-time big data processing in anomaly detection are discussed.

show abstract

Section: Random Forest (Rf)mentioning

confidence: 99%

Real-time big data processing for anomaly detection: A Survey

Habeeb¹,

Nasaruddin²,

Gani³

et al. 2019

International Journal of Information Management

273

View full text Add to dashboard Cite

show abstract

“…Tan et al [17] presented a prediction model of data flow based on linear autoregressive analysis and further proposed a real-time detection algorithm for outliers identification and compression processing. Fernandes et al [18] propose an autonomous profile-based anomaly detection system using principal component analysis and flow analysis to mitigate the impact of false data injection. By making inference of end-to-end measurements collected by relay nodes, Zheng et al [19] proposed a trust-assisted framework for detecting and localizing network anomalies in a hierarchical sensor network, which also can obtain a flexible tradeoff between inference accuracy and probing overhead.…”

Section: Related Workmentioning

confidence: 99%

“…υ 0 is a weighting exponent on each fuzzy membership that determines the amount of fuzziness of the resulting classification, and is set to 2. By using the Lagrange's multiplier to optimize formula (18), the problem is equivalent to find the minimum value of the Eq. (6).…”

Section: Cluster Formation Algorithmmentioning

confidence: 99%

Similarity-aware data aggregation using fuzzy c-means approach for wireless sensor networks

Wan

Xiong

et al. 2019

J Wireless Com Network

View full text Add to dashboard Cite

For resource-constrained IoT systems, data collection is one of the fundamental operations to reduce the energy dissipation of sensor nodes and improve the network lifetime. However, an anomaly or deviation will exert a great influence on the quality of data collected, especially for a data aggregation scheme. By taking into account dataaware clustering and detection of anomalous events, a similarity-aware data aggregation using a fuzzy c-means approach for wireless sensor networks is proposed. Firstly, by using a fuzzy c-means approach, the clustering process can be performed to organize sensors into clusters based on data similarity. Next, an effective support degree function is defined for further outlier diagnosis. Afterwards, the appropriate weight of valid data can be obtained by taking advantage of the probability distribution characteristics of normal samples within a certain period. Finally, the aggregation result in the cluster can be estimated. Practical database-based simulations have confirmed that the proposed data aggregation method can achieve better performance than traditional methods in terms of data outlier detection accuracy and relative recovery error.

show abstract

“…The need for such tools is based on our increasing dependence on computer systems, the growing convergence between physical and cyber systems discussed earlier, and recent events that demonstrate how nations use cyber warfare against each other. This, of course, does not disregard the advances in more traditional areas of cyber security, such as malware research, cryptography [Ganeshkumar et al 2014], early detection and intervention [Fernandes et al 2015], and privacy [Jentzsch 2015].…”

Section: Cyber Definitionmentioning

confidence: 99%