A Stochastic Proximal Alternating Minimization for Nonsmooth and Nonconvex Optimization

Driggs, Derek; Tang, Junqi; Liang, Jingwei; Davies, Mike E.; Schönlieb, Carola‐Bibiane

doi:10.1137/20m1387213

Cited by 15 publications

(7 citation statements)

References 21 publications

(4 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This proves the geometric decay of Γ k in expectation. Similar to Appendix B in (Driggs et al 2021), we also have that the third condition holds in Definition 4. This completes the proof.…”

Section: Proof Of Propositionmentioning

confidence: 86%

History of Mathematics in American Mathematical Textbooks of the Late 19th and Early 20th Centuries

Wang

2021

School Mathematics Textbooks in China

View full text Add to dashboard Cite

The logarithmic Schrödinger equation (LogSE) has a logarithmic nonlinearity f (u) = u ln |u| 2 that is not differentiable at u = 0. Compared with its counterpart with a regular nonlinear term, it possesses richer and unusual dynamics, though the low regularity of the nonlinearity brings about significant challenges in both analysis and computation. Among very limited numerical studies, the semi-implicit regularized method via regularising f (u) as u ε ln(ε+|u ε |) 2 to overcome the blowup of ln |u| 2 at u = 0 has been investigated recently in literature. With the understanding of f (0) = 0, we analyze the non-regularized first-order Implicit-Explicit (IMEX) scheme for the LogSE. We introduce some new tools for the error analysis that include the characterization of the Hölder continuity of the logarithmic term, and a nonlinear Grönwall's inequality. We provide ample numerical results to demonstrate the expected convergence. We position this work as the first one to study the direct linearized scheme for the LogSE as far as we can tell.

show abstract

Section: Proof Of Propositionmentioning

confidence: 86%

History of Mathematics in American Mathematical Textbooks of the Late 19th and Early 20th Centuries

Wang

2021

School Mathematics Textbooks in China

View full text Add to dashboard Cite

show abstract

“…1. Alternatively, we may also jointly optimize these parameters via alternating minimization (Bolte et al, 2014;Pock and Sabach, 2016;Driggs et al, 2021). 2.…”

Section: Algorithmic Frameworkmentioning

confidence: 99%

Data-Consistent Local Superresolution for Medical Imaging

Tang¹

2022

Preprint

Self Cite

View full text Add to dashboard Cite

In this work we propose a new paradigm of iterative model-based reconstruction algorithms for providing real-time solution for zooming-in and refining a region of interest in medical and clinical tomographic (such as CT/MRI/PET, etc) images. This algorithmic framework is tailor for a clinical need in medical imaging practice, that after a reconstruction of the full tomographic image, the clinician may believe that some critical parts of the image are not clear enough, and may wish to see clearer these regions-of-interest. A naive approach (which is highly not recommended) would be performing the global reconstruction of a higher resolution image, which has two major limitations: firstly, it is computationally inefficient, and secondly, the image regularization is still applied globally which may over-smooth some local regions. Furthermore if one wish to fine-tune the regularization parameter for local parts, it would be computationally infeasible in practice for the case of using global reconstruction. Our new iterative approaches for such tasks are based on jointly utilizing the measurement information, efficient upsampling/downsampling across image spaces, and locally adjusted image prior for efficient and high-quality post-processing. The numerical results in low-dose X-ray CT image local zoom-in demonstrate the effectiveness of our approach.

show abstract

“…To avoid such difficulty, many stochastic algorithms for non-convex problem involving three terms have also been proposed. In (Driggs et al, 2021), the authors presented a stochastic proximal alternating linearized minimization (called SPRING) algorithm which combines a class of variancereduced stochastic gradient estimators. In (Yurtsever et al, 2021), the authors extended DYS algorithm to a stochastic setting where the unbiased stochastic gradient estimators were considered.…”

Section: The Proposed Algorithm and Related Workmentioning

confidence: 99%

“…For solving the problem (2) with three-block structures, we propose a new stochastic alternating algorithm. Compared with the stochastic PALM (or SPRING) (Driggs et al, 2021), the SPRING algorithm solves the problem with H(x, y) being a finite sum in which the full gradients of H(x, y) are approximated using the variance-reduced gradient estimators. Our algorithm mainly focuses on the function G having large scale structure, thus we approximate the full gradient of G by unbiased stochastic gradient estimator.…”

Section: Contributionsmentioning

confidence: 99%

A stochastic three-block splitting algorithm and its application to quantized deep neural networks

Bian¹,

Liu²,

Zhang³

2022

Preprint

View full text Add to dashboard Cite

Deep neural networks (DNNs) have made great progress in various fields. In particular, the quantized neural network is a promising technique making DNNs compatible on resourcelimited devices for memory and computation saving. In this paper, we mainly consider a non-convex minimization model with three blocks to train quantized DNNs and propose a new stochastic three-block alternating minimization (STAM) algorithm to solve it. We develop a convergence theory for the STAM algorithm and obtain an -stationary point with optimal convergence rate O( −4 ). Furthermore, we apply our STAM algorithm to train DNNs with relaxed binary weights. The experiments are carried out on three different network structures, namely VGG-11, VGG-16 and ResNet-18. These DNNs are trained using two different data sets, CIFAR-10 and CIFAR-100, respectively. We compare our STAM algorithm with some classical efficient algorithms for training quantized neural networks. The test accuracy indicates the effectiveness of STAM algorithm for training relaxed binary quantization DNNs.

show abstract

A Stochastic Proximal Alternating Minimization for Nonsmooth and Nonconvex Optimization

Cited by 15 publications

References 21 publications

History of Mathematics in American Mathematical Textbooks of the Late 19th and Early 20th Centuries

History of Mathematics in American Mathematical Textbooks of the Late 19th and Early 20th Centuries

Data-Consistent Local Superresolution for Medical Imaging

A stochastic three-block splitting algorithm and its application to quantized deep neural networks

Contact Info

Product

Resources

About