An embedding approach to anomaly detection

Hu, Ruifeng; Aggarwal, Charų C.; Ma, Shuai; Huai, Jinpeng

doi:10.1109/icde.2016.7498256

Cited by 61 publications

(48 citation statements)

References 29 publications

(56 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figure 13: The anomalous (red) nodes in embedding, and A, B, C, D are four communities (Hu et al 2016). Image extracted from (Hu et al 2016). function φ parameterized by θ so as to bridge these two representations.…”

Section: Network Alignmentmentioning

confidence: 99%

“…Advanced information preserving network embedding Methods Source code Information diffusion (Bourigault et al 2014) https://github.com/ludc/social_network_diffusion_embeddings Cascade prediction https://github.com/chengli-um/DeepCas Anomaly detection (Hu et al 2016) https://github.com/hurenjun/EmbeddingAnomalyDetection Collaboration prediction (Chen and Sun 2017) https://github.com/chentingpc/GuidedHeteEmbedding direction is to explore the possibility of designing network embedding for more specific applications. For example, whether network embedding is a new way to detect rumors in social network (Seo, Mohapatra, and Abdelzaher 2012;Zhang et al 2015)?…”

Section: More Advanced Information and Tasksmentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Network Embedding

Cui

Wang

Pei

et al. 2019

IEEE Trans. Knowl. Data Eng.

1,033

559

View full text Add to dashboard Cite

Network embedding assigns nodes in a network to lowdimensional representations and effectively preserves the network structure. Recently, a significant amount of progresses have been made toward this emerging network analysis paradigm. In this survey, we focus on categorizing and then reviewing the current development on network embedding methods, and point out its future research directions. We first summarize the motivation of network embedding. We discuss the classical graph embedding algorithms and their relationship with network embedding. Afterwards and primarily, we provide a comprehensive overview of a large number of network embedding methods in a systematic manner, covering the structure-and property-preserving network embedding methods, the network embedding methods with side information and the advanced information preserving network embedding methods. Moreover, several evaluation approaches for network embedding and some useful online resources, including the network data sets and softwares, are reviewed, too. Finally, we discuss the framework of exploiting these network embedding methods to build an effective system and point out some potential future directions. IntroductionMany complex systems take the form of networks, such as social networks, biological networks, and information networks. It is well recognized that network data is often sophisticated and thus is challenging to deal with. To process network data effectively, the first critical challenge is to find effective network data representation, that is, how to represent networks concisely so that advanced analytic tasks, such as pattern discovery, analysis and prediction, can be conducted efficiently in both time and space.Traditionally, we usually represent a network as a graph G = V, E , where V is a vertex set representing the nodes in a network, and E is an edge set representing the relationships among the nodes. For large networks, such as those with billions of nodes, the traditional network representation poses several challenges to network processing and analysis.• High computational complexity. The nodes in a network are related to each other to a certain degree, encoded by the edge set E in the traditional network representation.

show abstract

Section: Network Alignmentmentioning

confidence: 99%

Section: More Advanced Information and Tasksmentioning

confidence: 99%

A Survey on Network Embedding

Cui

Wang

Pei

et al. 2019

IEEE Trans. Knowl. Data Eng.

1,033

559

View full text Add to dashboard Cite

show abstract

“…It was found that ego-networks with an equal number of nodes and edges is a network with a star-structure, were found to deviate from the derived power law [1]. Others attempted to improve the identification of nodes with star or clique structure by introducing alternate local features of the ego-network [9,14], incorporating features from the extended neighborhoods into the model [9,12], or by identifying so-called hubs, nodes that do not share a certain fraction of common neighbors with any of their adjacent nodes [24]. It should be noted that the node anomaly is considerably different from merely selecting high betweenness centrality nodes in the network, because this centrality measure does not differentiate between edges in or between different parts of the network, while node anomalies typically do [12].…”

Section: Related Workmentioning

confidence: 99%

A Community-Aware Approach for Identifying Node Anomalies in Complex Networks

Helling

Scholtes

Takes

2018

Studies in Computational Intelligence

View full text Add to dashboard Cite

The overwhelming amount of network data that is nowadays available, leads to an increased demand for techniques that automatically identify anomalous nodes. Examples are network intruders in physical networks or spammers spreading unwanted advertisements in online social networks. Existing methods typically identify network anomalies from a local perspective, only considering metrics related to a node and connections in its direct neighborhood. However, such methods often miss anomalies as they overlook crucial distortions of the network structure that are only visible at the macro level. To solve these problems, in this paper, the CADA algorithm is proposed, which identifies irregular nodes from a global perspective. It does so by measuring the extent to which a node connects to many different communities while not obviously belonging to one community itself. Results on synthetic and real-world data show that the incorporation of the community aspect is of critical importance, as our algorithm significantly outperforms previously suggested techniques. In addition, it scales well to larger networks of hundreds of thousands of nodes and millions of links. Moreover, the proposed method is parameter-free, enabling the hassle-free identification of anomalies in a wide variety of application domains.

show abstract

“…More recent work in this area looks into the attributed network by incorporating the vertex attributes [28,21,17]. Only a couple of recent works attempted to discover network outliers using network embeddings [6,11]. These efforts, however, do not apply to attributed networks.…”

mentioning

confidence: 99%

Semi-supervised Embedding in Attributed Networks with Outliers

Liang¹,

Jacobs²,

Sun³

et al. 2018

Proceedings of the 2018 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

In this paper, we propose a novel framework, called Semi-supervised Embedding in Attributed Networks with Outliers (SEANO), to learn a low-dimensional vector representation that systematically captures the topological proximity, attribute affinity and label similarity of vertices in a partially labeled attributed network (PLAN). Our method is designed to work in both transductive and inductive settings while explicitly alleviating noise effects from outliers. Experimental results on various datasets drawn from the web, text and image domains demonstrate the advantages of SEANO over the state-of-the-art methods in semi-supervised classification under transductive as well as inductive settings. We also show that a subset of parameters in SEANO are interpretable as outlier scores and can significantly outperform baseline methods when applied for detecting network outliers. Finally, we present the use of SEANO in a challenging real-world setting -flood mapping of satellite images and show that it is able to outperform modern remote sensing algorithms for this task. on the output outlier scores from SEANO and demonstrate its advantages over baseline methods specializing in network outlier detection. We finally conduct a case study of flood mapping to visually show the power of SEANO when applied to flood mapping. Related WorkNetwork Embedding: Network embedding strategies have gained increasing importance in recent years. Early ideas include IsoMap [35] and Locally Linear Embedding (LLE) [29], which exploited the manifold structure of vector data to compute low-dimensional embeddings. More recently, due to the emergence of naturally arising network data, other network embedding methods have been proposed [34,27,9]. In addition to learning embeddings for homogeneous networks, several researchers have proposed ideas for embedding attributed networks [36,13,12,26,37,15,10]. While they incorporate the attributes and/or label information into the embeddings, most are inherently transductive and cannot generate embeddings for vertices unseen during training. The two exceptions are Planetoid [37] and GraphSAGE [10] for inductive learning. However, Planetoid [37] is specialized for semi-supervised classification and the output network embedding, as a byproduct, does not capture all the information (as can be seen from their model architecture). Therefore, its embeddings might not be generalized to other applications such as visualization and clustering. GraphSAGE [10], on the other hand, only works on unsupervised learning or fully supervised learning setting and cannot be directly applied in a semi-supervised manner. Finally, none of the existing work on network embedding explicitly accounts for the impact of outliers. We summarize the differences between the proposed SEANO model with some of these recent efforts in Table 1. Method Attributes Labels Semisupervised Inductive Address Outliers DeepWalk [27] Node2Vec [9] TADW [36] LANE [13] TriDNR [26] Planetoid [37] GCN [15] GraphSAGE [10] SEANO Table 1: A comparison of SEANO w...

show abstract

An embedding approach to anomaly detection

Cited by 61 publications

References 29 publications

A Survey on Network Embedding

A Survey on Network Embedding

A Community-Aware Approach for Identifying Node Anomalies in Complex Networks

Semi-supervised Embedding in Attributed Networks with Outliers

Contact Info

Product

Resources

About