Lifelong representation learning in dynamic attributed networks

Wei, Hao; Hu, Guyu; Bai, Wei; Xia, Shiming; Pan, Zhisong

doi:10.1016/j.neucom.2019.05.038

Cited by 15 publications

(8 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…, where f is the number of additional node features, and learn representations leveraging these features in addition to their topological structure [7,50]. Although an edge feature matrix would be defined as well, its usage is much less common.…”

Section: Dynamic Graph With Additional Informationmentioning

confidence: 99%

“…additional information) that are changing over time in addition to the network structure and the topology may influence attribute modification. Therefore, embedding methods may also capture information evolution [50,56]. Furthermore, edge weights may also change over time in weighted networks, and their changes may be handled by embedding methods [60].…”

Section: Feature Evolutionmentioning

confidence: 99%

“…Different architectures based on neural networks are used both to extract the topological properties of a network and to capture temporal dependencies therein. Some preliminary works developed a fully-connected neural networks, either using them as an autoencoder or as a part of the decoding process [10,27,71,50]. Architectures based on recurrent neural networks (RNNs), including long shortterm memory units (LSTMs) [72] and gated recurrent units (GRUs) [73], leverage a sequence of graphs, or their representations, in order to learn or enhance embeddings taking into consideration temporal correlation, and store information over time to handle more complex correlations beyond consecutive timestamps [27,74,75].…”

Section: Approaches Based On Deep Learningmentioning

confidence: 99%

“…DynGEM [10] builds a fully-connected autoencoder for each graph snapshot, using a transfer learning paradigm to share parameters between two consecutive autoencoders and a strategy that allows the autoencoder network to widen its layers and inserts new layers in order to handle a growing number of nodes in the graph. LDANE [50] follows a similar strategy, and handles node attributes by adding a margin-based ranking loss term in the loss function that ensures the embeddings of two similar nodes are closer than the embedding of two non-similar nodes. Models employing recurrent neural networks and their hidden states to encode dynamic graph structure have also been proposed.…”

Section: Encoder-decoder Architecturementioning

confidence: 99%

See 3 more Smart Citations

A Survey on Embedding Dynamic Graphs

Barros¹,

Mendonça²,

Vieira³

et al. 2021

Preprint

View full text Add to dashboard Cite

Embedding static graphs in low-dimensional vector spaces plays a key role in network analytics and inference, supporting applications like node classification, link prediction, and graph visualization.However, many real-world networks present dynamic behavior, including topological evolution, feature evolution, and diffusion. Therefore, several methods for embedding dynamic graphs have been proposed to learn network representations over time, facing novel challenges, such as time-domain modeling, temporal features to be captured, and the temporal granularity to be embedded. In this survey, we overview dynamic graph embedding, discussing its fundamentals and the recent advances developed so far. We introduce the formal definition of dynamic graph embedding, focusing on the problem setting and introducing a novel taxonomy for dynamic graph embedding input and output.We further explore different dynamic behaviors that may be encompassed by embeddings, classifying by topological evolution, feature evolution, and processes on networks. Afterward, we describe existing techniques and propose a taxonomy for dynamic graph embedding techniques based on algorithmic approaches, from matrix and tensor factorization to deep learning, random walks, and temporal point processes. We also elucidate main applications, including dynamic link prediction, anomaly detection, and diffusion prediction, and we further state some promising research directions in the area.

show abstract

Section: Dynamic Graph With Additional Informationmentioning

confidence: 99%

Section: Feature Evolutionmentioning

confidence: 99%

Section: Approaches Based On Deep Learningmentioning

confidence: 99%

Section: Encoder-decoder Architecturementioning

confidence: 99%

See 2 more Smart Citations

A Survey on Embedding Dynamic Graphs

Barros¹,

Mendonça²,

Vieira³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The work in [45] trains sparse deep neural networks and compresses the sparse connections of each task in the network. The work in [48] expands the network for new tasks while preserving previous task knowledge and incorporating the topology and attribute of network nodes.…”

Section: Related Workmentioning

confidence: 99%

DILF-EN framework for Class-Incremental Learning

Asad¹,

Joshi²,

Mazumder³

et al. 2021

Preprint

View full text Add to dashboard Cite

A class-incremental learning problem is characterized by training data becoming available in a phase-by-phase manner. Deep learning models suffer from catastrophic forgetting of the classes in the older phases as they get trained on the classes introduced in the new phase. In this work, we show that the effect of catastrophic forgetting on the model prediction varies with the change in orientation of the same image, which is a novel finding. Based on this, we propose a novel data-ensemble approach that combines the predictions for the different orientations of the image to help the model retain further information regarding the previously seen classes and thereby reduce the effect of forgetting on the model predictions. However, we cannot directly use the data-ensemble approach if the model is trained using traditional techniques. Therefore, we also propose a novel dual-incremental learning framework that involves jointly training the network with two incremental learning objectives, i.e., the class-incremental learning objective and our proposed data-incremental learning objective. In the dualincremental learning framework, each image belongs to two classes, i.e., the image class (for class-incremental learning) and the orientation class (for data-incremental learning). In class-incremental learning, each new phase introduces a new set of classes, and the model cannot access the complete training data from the older phases. In our proposed data-incremental learning, the orientation classes remain the same across all the phases, and the data introduced by the new phase in class-incremental learning acts as new training data for these orientation classes. We empirically demonstrate that the dual-incremental learning framework is vital to the data-ensemble approach. We apply our proposed approach to state-of-the-art class-incremental learning methods and empirically show that our framework significantly improves the performance of these methods. Our proposed method significantly improves the performance of the stateof-the-art method (AANets) on the CIFAR-100 dataset by absolute margins of 3.30%, 4.28%, 3.55%, 4.03%, for the number of phases P=50, 25, 10, and 5, respectively, which establishes the efficacy of the proposed work.

show abstract

Design of a robust model predictive control for nonlinear switched systems: A low‐conservative and recursive feasibility strategy

Rahdan

Abedi

2023

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

This article proposes a robust H∞ based model predictive control scheme for Lipschitz nonlinear switched systems with persistent dwell time. In the developed method, the constraints corresponding to the recursive feasibility are included and the recursive feasibility is guaranteed. In this scheme, the existence of unstabilizable sub-systems and exogenous disturbances are included which makes it possible to avoid considering an ideal and simplistic assumption about switched systems. By simultaneously designing the controller and the switching signal and using multiple Lyapunov functions, the unacceptable conservatism in methods involving common Lyapunov functions and arbitrary switching signals is avoided. In the proposed solution, a time-varying prediction horizon is used for finite horizon cost functions, which in turn will be effective in reducing conservatism. In the suggested design strategy, the non-convex control scheme is formulated as an optimization problem in the framework of linear matrix inequality. Finally, a numerical example is developed and the performance of the proposed scheme is evaluated.

show abstract

Lifelong representation learning in dynamic attributed networks

Cited by 15 publications

References 10 publications

A Survey on Embedding Dynamic Graphs

A Survey on Embedding Dynamic Graphs

DILF-EN framework for Class-Incremental Learning

Design of a robust model predictive control for nonlinear switched systems: A low‐conservative and recursive feasibility strategy

Contact Info

Product

Resources

About