Network Dissensus via Distributed ADMM

Kumar, Chirag; Rajawat, Ketan

doi:10.1109/tsp.2020.2984895

Cited by 5 publications

(3 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ADMM is a popular technique used for solving convex optimization problems in machine learning and deep leaning, making possible a large-scale optimization [26]. Recent works also demonstrate that under certain conditions, the ADMM is guaranteed to converge for non-convex problems [27]. Specifically, the ADMM can separate the variables and decompose the problem into two subproblems.…”

Section: Preliminary: Admmmentioning

confidence: 99%

Compression of Deep-Learning Models Through Global Weight Pruning Using Alternating Direction Method of Multipliers

Lee

Hwangbo

Yang

et al. 2023

Int J Comput Intell Syst

View full text Add to dashboard Cite

Deep learning has shown excellent performance in numerous machine-learning tasks, but one practical obstacle in deep learning is that the amount of computation and required memory is huge. Model compression, especially in deep learning, is very useful because it saves memory and reduces storage size while maintaining model performance. Model compression in a layered network structure aims to reduce the number of edges by pruning weights that are deemed unnecessary during the calculation. However, existing weight pruning methods perform a layer-by-layer reduction, which requires a predefined removal-ratio constraint for each layer. Layer-by-layer removal ratios must be structurally specified depending on the task, causing a sharp increase in the training time due to a large number of tuning parameters. Thus, such a layer-by-layer strategy is hardly feasible for deep layered models. Our proposed method aims to perform weight pruning in a deep layered network, while producing similar performance, by setting a global removal ratio for the entire model without prior knowledge of the structural characteristics. Our experiments with the proposed method show reliable and high-quality performance, obviating layer-by-layer removal ratios. Furthermore, experiments with increasing layers yield a pattern in the pruned weights that could provide an insight into the layers’ structural importance. The experiment with the LeNet-5 model using MNIST data results in a higher compression ratio of 98.8% for the proposed method, outperforming existing pruning algorithms. In the Resnet-56 experiment, the performance change according to removal ratios of 10–90% is investigated, and a higher removal ratio is achieved compared to other tested models. We also demonstrate the effectiveness of the proposed method with YOLOv4, a real-life object-detection model requiring substantial computation.

show abstract

Section: Preliminary: Admmmentioning

confidence: 99%

Compression of Deep-Learning Models Through Global Weight Pruning Using Alternating Direction Method of Multipliers

Lee

Hwangbo

Yang

et al. 2023

Int J Comput Intell Syst

View full text Add to dashboard Cite

show abstract

“…To solve this massive scalability challenge while addressing privacy, latency, reliability and bandwidth efficiency, distributed learning frameworks [19]- [23], e.g., federated learning (FL) [24]- [26] and MapReduce [44], are needed, and consequently intelligence must be pushed to the network edge in future communication systems with optimization algorithms, e.g., alternating direction method of multipliers (ADMM) [27], [28] and distributed gradient descend [29]. In these frameworks, communication units/devices/nodes are capable of collaboratively building a shared learning model with training their collected data locally.…”

Section: Introductionmentioning

confidence: 99%

“…Similarly, to increase robustness, ADMM is widely considered for large scale distributed learning. Likewise, distributed gradient descend methods are also studied for various potential applications [27], [28].…”

Section: Introductionmentioning

confidence: 99%

Distributed Learning for Wireless Communications: Methods, Applications and Challenges

Qian

Yang

Xiao

et al. 2022

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

With its privacy-preserving and decentralized features, distributed learning plays an irreplaceable role in the era of wireless networks with a plethora of smart terminals, an explosion of information volume and increasingly sensitive data privacy issues. There is a tremendous increase in the number of scholars investigating how distributed learning can be employed to emerging wireless network paradigms in the physical layer, media access control layer and network layer. Nonetheless, researches on distributed learning for wireless communications are still in its infancy. In this paper, we review the contemporary technical applications of distributed learning for wireless communications. We first introduce the typical frameworks and algorithms for distributed learning. Examples of applications of distributed learning frameworks in the emerging wireless network paradigms are then provided. Finally, main research directions and challenges of distributed learning for wireless communications are discussed.

show abstract

An ADMM approach for elliptic positioning in non-line-of-sight environments

Xiong,

He,

et al. 2024

Digital Signal Processing

View full text Add to dashboard Cite

Network Dissensus via Distributed ADMM

Cited by 5 publications

References 69 publications

Compression of Deep-Learning Models Through Global Weight Pruning Using Alternating Direction Method of Multipliers

Compression of Deep-Learning Models Through Global Weight Pruning Using Alternating Direction Method of Multipliers

Distributed Learning for Wireless Communications: Methods, Applications and Challenges

An ADMM approach for elliptic positioning in non-line-of-sight environments

Contact Info

Product

Resources

About