Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks

Abstract

Modeling sequential interactions between users and items/products is crucial in domains such as e-commerce, social networking, and education. Representation learning presents an attractive opportunity to model the dynamic evolution of users and items, where each user/item can be embedded in a Euclidean space and its evolution can be modeled by an embedding trajectory in this space. However, existing dynamic embedding methods generate embeddings only when users take actions and do not explicitly model the future trajectory of the user/item in the embedding space. Here we propose JODIE, a coupled recurrent neural network model that learns the embedding trajectories of users and items. JODIE employs two recurrent neural networks to update the embedding of a user and an item at every interaction. Crucially, JODIE also models the future embedding trajectory of a user/item. To this end, it introduces a novel projection operator that learns to estimate the embedding of the user at any time in the future. These estimated embeddings are then used to predict future user-item interactions. To make the method scalable, we develop a t-Batch algorithm that creates time-consistent batches and leads to 9× faster training. We conduct six experiments to validate JODIE on two prediction tasks— future interaction prediction and state change prediction—using four real-world datasets. We show that JODIE outperforms six state-of-the-art algorithms in these tasks by at least 20% in predicting future interactions and 12% in state change prediction.

Continuous-Time Dynamic Graph Learning via Neural Interaction Processes

“…In DyRep [22], both association and communication interactions are modeled by a two time-scale TPPs. A recent method JODIE [11] also employs the coupled RNNs to generate dynamic embeddings but trained by predicting the next embedding directly instead of modeling the intensity. In this work, unlike previous methods that use the coupled vanilla RNNs and have limited model capacity, our dynamic message passing neural network(TDIG-MPNN) can generate more expressive and informative embedding representations.…”

“…Relation to previous work. The graph construction in the Deep-Coevolveve and JODIE [2,11] differs from our in that the nodes ( ) and ( ) associated with the , , are only connected by these two nodes themselves before the time (i.e., they use a different way to construct the temporal graph edges and they don't provide a formal definition about their graph), which means when updating the states, only previous states of these two nodes themselves are considered, not including the states of the other parties is observed at time 6 , the corresponding TDIG is updated with one newly-added two-way interaction edge and the four newly-added directed dependency edges, i.e., 0 , 1 , 2 and 3 . The red dotted square denotes the 3-depth TDIG sub-graph G 6 ( , = 3) for the node at 6 and the Δ ( , 6 ) represents the time interval(as the edge feature for the edge 3 and 4 ) between , , 6 and the last interaction "Lucy saw the Skull movie at 5 " where the node was involved.…”

“…They used the coupled recurrent neural networks (RNNs) to learn dynamic embeddings and define different neural intensity functions of TPPs to characterize dynamic processes. Recently, JODIE [11] also employs the coupled RNNs to generate dynamic embeddings but it is trained by predicting the next embedding directly instead of modeling the intensity. The main drawback of the existing coupled-RNNs-based methods is: regarding to the current interaction(nodes), they treat all the historical interaction information equally and utilize vanilla coupled RNNs to compress them into compact representations as dynamic embeddings of the current nodes.…”

Learning Semantic Relationships of Geographical Areas based on Trajectories

“…Similarly in [8], authors discuss how to enrich trajectories with semantic information based on stop points in moving objects. Kumar et al [16] proposed a model that learns dynamic trajectory embeddings of users and items from a sequence of temporal interactions. We look at trajectories and geographical areas in terms of developing and enriching semantic understanding of regions.…”

Higher-Order Label Homogeneity and Spreading in Graphs

“…Graph SSL finds applications in a number of settings: in a social network, we can infer a particular characteristic (e.g. political leaning) of a user based on the information of her friends to produce tailored recommendations; in a user-product bipartite rating network, based on a few manually identified fraudulent user accounts, SSL is useful to spot other fraudulent accounts [4,10,18,19]; SSL can identify protein functions from networks of their physical interaction using just a few labels [32].…”