APAN: Asynchronous Propagation Attention Network for Real-time Temporal Graph Embedding

TGL: A General Framework for Temporal GNN Training on Billion-Scale Graphs

“…Table 1 shows different strategies used by different TGNN variants. Note that some TGNNs [15,23] use combinations of multiple strategies to intensify the temporal relationships, while some other TGNNs only rely on a single strategy. For example, pure memory TGNNs [10,18,23] directly use the node memory as the dynamic node embeddings, potentially with complex COMB and UPDT function to update node memory.…”

“…Note that some TGNNs [15,23] use combinations of multiple strategies to intensify the temporal relationships, while some other TGNNs only rely on a single strategy. For example, pure memory TGNNs [10,18,23] directly use the node memory as the dynamic node embeddings, potentially with complex COMB and UPDT function to update node memory. For example, in APAN [23], the mails are delivered to the mailboxes of hop-1 neighbors and the COMB function applies attention mechanism to update the node memory.…”

“…For example, pure memory TGNNs [10,18,23] directly use the node memory as the dynamic node embeddings, potentially with complex COMB and UPDT function to update node memory. For example, in APAN [23], the mails are delivered to the mailboxes of hop-1 neighbors and the COMB function applies attention mechanism to update the node memory. After studying the architecture of different TGNNs, we identify three components that form a unified representation for most TGNN variants -the node memory, the attention aggregator, and the temporal sampler.…”

“…In node memory, the information leak problem is eliminated by caching the input from previous mini-batches [15], which enables the node memory to receive gradients. In TGL, we adopt the mailbox module [23] to store a fixed number of most recent mails for updating the node memory. When a new interaction appears, we first update the node memory of the involved nodes with the cached messages in the mailbox.…”

“…• We design a unified framework that supports efficient training on most TGNN architectures by studying the characteristic of a diverse set of TGNNs variants including snapshotbased TGNNs [13,17], time encoding-based TGNNs [1,15,23], and memory-based TGNNs [10,15,18,23].…”