Learning Dynamic Embeddings from Temporal Interactions

Abstract

Modeling a sequence of interactions between users and items (e.g., products, posts, or courses) is crucial in domains such as e-commerce, social networking, and education to predict future interactions. Representation learning presents an attractive solution to model the dynamic evolution of user and item properties, where each user/item can be embedded in a euclidean space and its evolution can be modeled by dynamic changes in its embedding. However, existing embedding methods either generate static embeddings, treat users and items independently, or are not scalable.Here we propose JODIE, a coupled recurrent model to jointly learn the dynamic embeddings of users and items from a sequence of user-item interactions. JODIE has three components. First, the update component updates the user and item embedding from each interaction using their previous embeddings with the two mutuallyrecursive Recurrent Neural Networks. Second, a novel projection component is trained to forecast the embedding of users at any future time. Finally, the prediction component directly predicts the embedding of the item in a future interaction. For models that learn from a sequence of interactions, traditional batching of training data can not be done due to complex user-user dependencies. Therefore, we present a novel batching algorithm called t-Batch that generates time-consistent batches that can be run in parallel, leading to massive speed-up.We conduct six experiments to validate JODIE on two prediction tasks-future interaction prediction and state change predictionusing four real-world datasets. We show that JODIE outperforms six state-of-the-art algorithms in these tasks by up to 22.4%. Moreover, we show that JODIE is highly scalable and up to 9.2× faster than comparable models. As an additional experiment, we illustrate that JODIE can predict student drop-out from courses up to five interactions in advance.

Dynamic Hidden-Variable Network Models

“…One direct real-world application of THVMs could be to serve as null models [65,181] for evolving networks with dynamic node-properties [75]. Dynamic embedding methods [142][143][144][145][146][147][148][149][150][151][152][153][154], or generalizations of inference methods from dynamic SBMs [73], could potentially allow retrieval of H (and perhaps also σ, ω, and f ) from an observed G. Links of real evolving networks may not in general be fully equilibrated relative to the current set of nodecharacteristics, which is a dynamical behavior exhibited by THVMs outside of the quasi-static regime. Hence in some cases, the Equilibrium Property and Qualitative Realism may be in conflict, implying that caution should be used when applying static models to snapshots of evolving networks.…”

“…Another study was of a temporal hyper-SBM with ω < 1 which thus exhibits both link-persistence and group-assignment-persistence [73], influencing performance of community detection algorithms and motivating the development of new ones. Another area of relevant work is the rapidly emerging area of dynamic graph embeddings [75,[142][143][144][145][146][147][148][149][150][151][152][153][154], related to the task of inference of hidden-variable trajectories [155].…”

Learning Student Interest Trajectory for MOOCThread Recommendation

“…RNNs have been used for modeling the evolving features of items and users in [7], [11]. These models, similar to ours, also update the state of users and items after they interact.…”

“…Similar to JODIE model [11], we predict the embedding of the next thread that will interest the student. We make this prediction using the projected student embedding û(t + ∆) and the embedding of thread p(t) of thread p (the thread on which u last posted on).…”

Efficient Scaling of Dynamic Graph Neural Networks

“…. , G T , where T is the number of timesteps and G t is the graph as it stood at timestep t. Various models have been designed for learning within both the CTDG (e.g., [18,12,24,15,20]) and the DTDG (e.g., [3,22,17,13,19,16]) frameworks. We refer to the survey by Kazemi et al [9] for a detailed discussion on the topic.…”