Kernel Ridge Regression-Based Graph Dataset Distillation

Xu, Zhe; Chen, Yuzhong; Pan, Menghai; Chen, Huiyuan; Das, Mahashweta; Yang, Hao; Tong, Hanghang

doi:10.1145/3580305.3599398

Cited by 7 publications

(2 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Matrix factorization is an early approach to learn the latent embeddings of users and items and use the inner product to predict the users' preference [19]. Motivated by the success of deep neural networks [5,12,16,36,41,44,46], deep recommender systems can be further used to exploit more complex and nonlinear feature interactions between users and items. Some popular models include NCF [16], DeepFM [12], DIN [53], etc.…”

Section: Related Work 21 Collaborative Filteringmentioning

confidence: 99%

Towards Mitigating Dimensional Collapse of Representations in Collaborative Filtering

Chen,

Lai,

Jin

et al. 2024

Proceedings of the 17th ACM International Conference on Web Search and Data Mining

Self Cite

View full text Add to dashboard Cite

Contrastive Learning (CL) has shown promising performance in collaborative filtering. The key idea is to generate augmentationinvariant embeddings by maximizing the Mutual Information between different augmented views of the same instance. However, we empirically observe that existing CL models suffer from the dimensional collapse issue, where user/item embeddings only span a low-dimension subspace of the entire feature space. This suppresses other dimensional information and weakens the distinguishability of embeddings. Here we propose a non-contrastive learning objective, named nCL, which explicitly mitigates dimensional collapse of representations in collaborative filtering. Our nCL aims to achieve geometric properties of Alignment and Compactness on the embedding space. In particular, the alignment tries to push together representations of positive-related user-item pairs, while compactness tends to find the optimal coding length of user/item embeddings, subject to a given distortion. More importantly, our nCL does not require data augmentation nor negative sampling during training, making it scalable to large datasets. Experimental results demonstrate the superiority of our nCL. CCS CONCEPTS• Information systems → Recommender systems.

show abstract

Section: Related Work 21 Collaborative Filteringmentioning

confidence: 99%

Towards Mitigating Dimensional Collapse of Representations in Collaborative Filtering

Chen,

Lai,

Jin

et al. 2024

Proceedings of the 17th ACM International Conference on Web Search and Data Mining

Self Cite

View full text Add to dashboard Cite

show abstract

“…For example, the FP32 embeddings of 10 million items with a dimensional size of 256 will take up over 9.5 GB of storage space, which is hard to be deployed into devices with limited memory, especially under the federated learning settings [25,37]. Therefore, searching and ranking from a large item corpus to generate top-𝑘 recommendations become intractable at scale due to their high latency [4,26,28,29,33].…”

Section: Introductionmentioning

confidence: 99%

Hessian-aware Quantized Node Embeddings for Recommendation

Chen,

Zhou,

Lai

et al. 2023

Proceedings of the 17th ACM Conference on Recommender Systems

Self Cite

View full text Add to dashboard Cite

Graph Neural Networks (GNNs) have achieved state-of-the-art performance in recommender systems. Nevertheless, the process of searching and ranking from a large item corpus usually requires high latency, which limits the widespread deployment of GNNs in industry-scale applications. To address this issue, many methods compress user/item representations into the binary embedding space to reduce space requirements and accelerate inference. Also, they use the Straight-through Estimator (STE) to prevent vanishing gradients during back-propagation. However, the STE often causes the gradient mismatch problem, leading to sub-optimal results.In this work, we present the Hessian-aware Quantized GNN (HQ-GNN) as an effective solution for discrete representations of users/items that enable fast retrieval. HQ-GNN is composed of two components: a GNN encoder for learning continuous node embeddings and a quantized module for compressing full-precision embeddings into low-bit ones. Consequently, HQ-GNN benefits from both lower memory requirements and faster inference speeds compared to vanilla GNNs. To address the gradient mismatch problem in STE, we further consider the quantized errors and its secondorder derivatives for better stability. The experimental results on several large-scale datasets show that HQ-GNN achieves a good balance between latency and performance. CCS CONCEPTS• Information systems → Recommender systems; • Computing methodologies → Neural networks.

show abstract

Dancing with Still Images: Video Distillation via Static-Dynamic Disentanglement

Wang,

Xu,

et al. 2024

2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Kernel Ridge Regression-Based Graph Dataset Distillation

Cited by 7 publications

References 16 publications

Towards Mitigating Dimensional Collapse of Representations in Collaborative Filtering

Towards Mitigating Dimensional Collapse of Representations in Collaborative Filtering

Hessian-aware Quantized Node Embeddings for Recommendation

Dancing with Still Images: Video Distillation via Static-Dynamic Disentanglement

Contact Info

Product

Resources

About