Multi-dimensional balanced graph partitioning via projected gradient descent

Avdiukhin, Dmitrii; Pupyrev, Sergey; Yaroslavtsev, Grigory

doi:10.14778/3324301.3324307

Cited by 14 publications

(5 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To train a GNN, we rely on the technique called message passing, we refer the readers to Appendix A of [10] for more details. 2…”

Section: Preliminaries 21 Graph Neural Networkmentioning

confidence: 99%

“…To address this issue, after each gradient descent iteration, we map the negative importance score back to 0. The mapping the negative importance scores to 0 follows the general idea of projected gradient descent (PGD) [2]. In addition, the summation of the importance scores could deviate from 1.…”

Section: Learning-based Aggregation (Lbaggr)mentioning

confidence: 99%

“…A promising solution for large-scale graph partitioning is utilizing the multilevel graph partitioning methods. The general idea is first to contract edges and obtain smaller graphs, then cut the resulting graph, and finally unfold back to the original graph with some local improvement criterion [1,2,28]. Among the multilevel graph partitioning methods, METIS [31,33] is a family of the most widely known techniques and achieves state-of-the-art performance [3].…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Graph Unlearning

Chen

Zhang

Wang

et al. 2022

Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security

View full text Add to dashboard Cite

Machine unlearning is a process of removing the impact of some training data from the machine learning (ML) models upon receiving removal requests. While straightforward and legitimate, retraining the ML model from scratch incurs a high computational overhead. To address this issue, a number of approximate algorithms have been proposed in the domain of image and text data, among which SISA is the state-of-the-art solution. It randomly partitions the training set into multiple shards and trains a constituent model for each shard. However, directly applying SISA to the graph data can severely damage the graph structural information, and thereby the resulting ML model utility. In this paper, we propose GraphEraser, a novel machine unlearning framework tailored to graph data. Its contributions include two novel graph partition algorithms and a learning-based aggregation method. We conduct extensive experiments on five real-world graph datasets to illustrate the unlearning efficiency and model utility of GraphEraser. It achieves 2.06× (small dataset) to 35.94× (large dataset) unlearning time improvement. On the other hand, GraphEraser achieves up to 62.5% higher F1 score and our proposed learning-based aggregation method achieves up to 112% higher F1 score. 1 CCS CONCEPTS• Security and privacy; • Computing methodologies → Machine learning;

show abstract

“…To train a GNN, we rely on the technique called message passing, we refer the readers to Appendix A of [10] for more details. 2…”

Section: Preliminaries 21 Graph Neural Networkmentioning

confidence: 99%

Section: Learning-based Aggregation (Lbaggr)mentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Graph Unlearning

Chen

Zhang

Wang

et al. 2022

Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security

View full text Add to dashboard Cite

show abstract

“…Scaling up B++&C. One of the main advantages of B++&C is its efficiency. The algorithm is a gradient descent approach inspired by Avdiukhin, Pupyrev, and Yaroslavtsev (2019) applied to a quadratic function.Using a technique which we refer to as inverse kernel trick (see Section 2), for several widely used similarities and distance measures, we can represent A as a product of low-rank matrices, which allows us to compute the gradient efficiently.…”

Section: Our Contributionsmentioning

confidence: 99%

Objective-Based Hierarchical Clustering of Deep Embedding Vectors

Naumov

Yaroslavtsev

Avdiukhin

2021

AAAI

Self Cite

View full text Add to dashboard Cite

We initiate a comprehensive experimental study of objective-based hierarchical clustering methods on massive datasets consisting of deep embedding vectors from computer vision and NLP applications. This includes a large variety of image embedding (ImageNet, ImageNetV2, NaBirds), word embedding (Twitter, Wikipedia), and sentence embedding (SST-2) vectors from several popular recent models (e.g. ResNet, ResNext, Inception V3, SBERT). Our study includes datasets with up to 4.5 million data points with embedding dimensions up to 2048. In order to address the challenge of scaling up hierarchical clustering to such large datasets we propose a new practical hierarchical clustering algorithm B++&C. It gives a 5%/20% improvement on average for the popular Moseley-Wang (MW)/ Cohen-Addad et al. (CKMM) objectives (normalized) compared to a wide range of classic methods and recent heuristics. We also introduce a theoretical algorithm B2SAT&C which achieves a 0.74-approximation for the CKMM objective in polynomial time. This is the first substantial improvement over the trivial 2/3-approximation achieved by a random binary tree. Prior to this work, the best poly-time approximation of ≈2/3 + 0.0004 was due to Charikar et al. (SODA’19)

show abstract

“…Our algorithm is based on the approach described in Avdiukhin, Pupyrev, and Yaroslavtsev [2019]: we optimize a continuous relaxation (x i ∈ [−1, 1]) of the function above. Algorithm 1 is a projected gradient descent approach which optimizes f (x) = x ⊤ W x under constraints x i ∈ [−1, 1] and i x i = 2δn.…”

Section: Bisect++ and Conquermentioning

confidence: 99%

Objective-Based Hierarchical Clustering of Deep Embedding Vectors

Naumov¹,

Yaroslavtsev²,

Avdiukhin³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

We initiate a comprehensive experimental study of objective-based hierarchical clustering methods on massive datasets consisting of deep embedding vectors from computer vision and NLP applications. This includes a large variety of image embedding (ImageNet, ImageNetV2, NaBirds), word embedding (Twitter, Wikipedia), and sentence embedding (SST-2) vectors from several popular recent models (e.g. ResNet, ResNext, Inception V3, SBERT). Our study includes datasets with up to 4.5 million entries with embedding dimensions up to 2048.In order to address the challenge of scaling up hierarchical clustering to such large datasets we propose a new practical hierarchical clustering algorithm B++&C. It gives a 5%/20% improvement on average for the popular Moseley-Wang (MW) / Cohen-Addad et al. (CKMM) objectives (normalized) compared to a wide range of classic methods and recent heuristics. We also introduce a theoretical algorithm B2SAT&C which achieves a 0.74-approximation for the CKMM objective in polynomial time. This is the first substantial improvement over the trivial 2/3-approximation achieved by a random binary tree. Prior to this work, the best poly-time approximation of ≈ 2/3 + 0.0004 was due to Charikar et al. (SODA'19).

show abstract

Multi-dimensional balanced graph partitioning via projected gradient descent

Cited by 14 publications

References 35 publications

Graph Unlearning

Graph Unlearning

Objective-Based Hierarchical Clustering of Deep Embedding Vectors

Objective-Based Hierarchical Clustering of Deep Embedding Vectors

Contact Info

Product

Resources

About