Neural Fixed-Point Acceleration for Convex Optimization

Venkataraman, Shobha; Amos, Brandon

doi:10.48550/arxiv.2107.10254

Cited by 2 publications

(4 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to the development of the methodologies, another trend is to explore new applications in wireless networks. Besides the optimization problems mentioned before, learning to optimize techniques are expected to solve other problems involved in wireless communication for future research directions, including multi-object optimization problems [245], bi-level optimization problems [246], conic programming [247], maximum-likelihood estimation problems [248], etc, in various emerging applications.…”

Section: ) Advanced Methodologies and Extended Applications Of Moasmentioning

confidence: 99%

Machine Learning for Large-Scale Optimization in 6G Wireless Networks

Shi¹,

Lian²,

Shi³

et al. 2023

Preprint

View full text Add to dashboard Cite

The sixth generation (6G) wireless systems are envisioned to enable the paradigm shift from "connected things" to "connected intelligence", featured by ultra high density, largescale, dynamic heterogeneity, diversified functional requirements and machine learning capabilities, which leads to a growing need for highly efficient intelligent algorithms. The classic optimization-based algorithms usually require highly precise mathematical model of data links and suffer from poor performance with high computational cost in realistic 6G applications. Based on domain knowledge (e.g., optimization models and theoretical tools), machine learning (ML) stands out as a promising and viable methodology for many complex large-scale optimization problems in 6G, due to its superior performance, generalizability, computational efficiency and robustness. In this paper, we systematically review the most representative "learning to optimize" techniques in diverse domains of 6G wireless networks by identifying the inherent feature of the underlying optimization problem and investigating the specifically designed ML frameworks from the perspective of optimization. In particular, we will cover algorithm unrolling, learning to branch-andbound, graph neural network for structured optimization, deep reinforcement learning for stochastic optimization, as well as endto-end learning for semantic optimization, for solving challenging large-scale optimization problems arising from various important wireless applications. To enable ML implementation in distributed wireless networks across massive number of end devices, federated learning for distributed optimization will further be presented. Through the in-depth discussion, we shed light on the excellent performance of ML-based optimization algorithms with respect to the classical methods, and provide insightful guidance to develop advanced ML techniques in 6G networks. Neural network design, theoretical tools of different ML methods,

show abstract

Section: ) Advanced Methodologies and Extended Applications Of Moasmentioning

confidence: 99%

Machine Learning for Large-Scale Optimization in 6G Wireless Networks

Shi¹,

Lian²,

Shi³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…• Section 6.4.1 discusses models that integrate fixed-point computations into semiamortized models. Venkataraman and Amos (2021) amortize convex cone programs by differentiating through the splitting cone solver (O'donoghue et al, 2016) and Bai et al (2022) amortize deep equilibrium models (Bai et al, 2019(Bai et al, , 2020.…”

Section: Semi-amortized Modelsmentioning

confidence: 99%

“…Neural fixed-point acceleration (Venkataraman and Amos, 2021) proposes a semi-amortized method for computing fixed-points and use it for convex cone programming. Representing a latent state at time t with ĥt , they parameterize the initial iterate ŷ0 , ĥ0 = init θ (x) with an initialization model init θ and perform the fixed-point computations…”

Section: Neural Fixed-point Acceleration (Neuralfp) and Conic Optimiz...mentioning

confidence: 99%

See 1 more Smart Citation

Tutorial on amortized optimization for learning to optimize over continuous domains

Amos¹

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Optimization is a ubiquitous modeling tool and is often deployed in settings which repeatedly solve similar instances of the same problem. Amortized optimization methods use learning to predict the solutions to problems in these settings. This leverages the shared structure between similar problem instances. In this tutorial, we will discuss the key design choices behind amortized optimization, roughly categorizing 1) models into fully-amortized and semi-amortized approaches, and 2) learning methods into regression-based and objectivebased. We then view existing applications through these foundations to draw connections between them, including for manifold optimization, variational inference, sparse coding, meta-learning, control, reinforcement learning, convex optimization, and deep equilibrium networks. This framing enables us easily see, for example, that the amortized inference in variational autoencoders is conceptually identical to value gradients in control and reinforcement learning as they both use fully-amortized models with an objective-based loss.

show abstract

Neural Fixed-Point Acceleration for Convex Optimization

Cited by 2 publications

References 22 publications

Machine Learning for Large-Scale Optimization in 6G Wireless Networks

Machine Learning for Large-Scale Optimization in 6G Wireless Networks

Tutorial on amortized optimization for learning to optimize over continuous domains

Contact Info

Product

Resources

About