Deep Neural Network Structures Solving Variational Inequalities

Combettes, Patrick L.; Pesquet, Jean-Christophe

doi:10.1007/s11228-019-00526-z

Cited by 94 publications

(106 citation statements)

References 39 publications

Supporting

Mentioning

104

Contrasting

Order By: Relevance

“…, K − 1}, R k (· + b k ) is firmly nonexpansive [38,Proposition 12.28]. Finally, [59,Theorem 3.8] completes the proof.…”

Section: Robustness Of Irestnet To An Input Perturbationmentioning

confidence: 81%

“…Before stating our main stability theorem, we recall the result from [59,Lemma 3.3] in Proposition 4 below. We then derive Proposition 5, which will appear useful when addressing the robustness of the global network.…”

Section: Preliminary Resultsmentioning

confidence: 99%

“…If one of the conditions (i)-(iii) is satisfied, then we deduce from Proposition 5 that W satisfies [59, Proposition 3.6(iii)]. and [59,Condition 3.1]. In addition, for every k ∈ {0, .…”

Section: Robustness Of Irestnet To An Input Perturbationmentioning

confidence: 92%

“…For example, the authors of [15] show that the class prediction made by AlexNet can be arbitrarily changed by using small nonrandom perturbations on the test image. A recent work [59] provides a theoretical framework which enables to evaluate the robustness of a network. In this section, we will focus on a subclass of problem (2) where both f (·, y) and R are quadratic functions.…”

Section: Network Stabilitymentioning

confidence: 99%

“…where (W k ) 0≤k≤K−1 and (b k ) 0≤k≤K−1 are interpreted as weight operators and bias parameters, respectively. The operators (R k ) 0≤k≤K−1 defined in (61) can be viewed as specific activation functions since, as shown in[59], every standard activation function can be derived from a proximity operator. In addition, using[38, Proposition 24.8(iii)], for every k ∈ {0, .…”

mentioning

confidence: 99%

See 4 more Smart Citations

Deep unfolding of a proximal interior point method for image restoration

Bertocchi¹,

Chouzenoux²,

Corbineau³

et al. 2020

Inverse Problems

105

View full text Add to dashboard Cite

Variational methods are widely applied to ill-posed inverse problems for they have the ability to embed prior knowledge about the solution. However, the level of performance of these methods significantly depends on a set of parameters, which can be estimated through computationally expensive and timeconsuming methods. In contrast, deep learning offers very generic and efficient architectures, at the expense of explainability, since it is often used as a black-box, without any fine control over its output. Deep unfolding provides a convenient approach to combine variational-based and deep learning approaches. Starting from a variational formulation for image restoration, we develop iRestNet, a neural network architecture obtained by unfolding a proximal interior point algorithm. Hard constraints, encoding desirable properties for the restored image, are incorporated into the network thanks to a logarithmic barrier, while the barrier parameter, the stepsize, and the penalization weight are learned by the network. We derive explicit expressions for the gradient of the proximity operator for various choices of constraints, which allows training iRestNet with gradient descent and backpropagation. In addition, we provide theoretical results regarding the stability of the network for a common inverse problem example. Numerical experiments on image deblurring problems show that the proposed approach compares favorably with both state-of-the-art variational and machine learning methods in terms of image quality.

show abstract

“…, K − 1}, R k (· + b k ) is firmly nonexpansive [38,Proposition 12.28]. Finally, [59,Theorem 3.8] completes the proof.…”

Section: Robustness Of Irestnet To An Input Perturbationmentioning

confidence: 81%

Section: Preliminary Resultsmentioning

confidence: 99%

“…If one of the conditions (i)-(iii) is satisfied, then we deduce from Proposition 5 that W satisfies [59, Proposition 3.6(iii)]. and [59,Condition 3.1]. In addition, for every k ∈ {0, .…”

Section: Robustness Of Irestnet To An Input Perturbationmentioning

confidence: 92%

Section: Network Stabilitymentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Deep unfolding of a proximal interior point method for image restoration

Bertocchi¹,

Chouzenoux²,

Corbineau³

et al. 2020

Inverse Problems

105

View full text Add to dashboard Cite

show abstract

Variational Models for Signal Processing with Graph Neural Networks

Azad

Rabin

Elmoataz

2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Safe Design of Stable Neural Networks for Fault Detection in Small UAVs

Gupta

Kaakai

Pesquet‐Popescu

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Stability of a machine learning model is the extent to which a model can continue to operate correctly despite small perturbations in its inputs. A formal method to measure stability is the Lipschitz constant of the model which allows to evaluate how small perturbations in the inputs impact the output variations. Variations in the outputs may lead to high errors for regression tasks or unintended changes in the classes for classification tasks. Verification of the stability of ML models is crucial in many industrial domains such as aeronautics, space, automotive etc. It has been recognized that data-driven models are intrinsically extremely sensitive to small perturbation of the inputs. Therefore, the need to design methods for verifying the stability of ML models is of importance for manufacturers developing safety critical products. In this work, we focus on Small Unmanned Aerial Vehicles (UAVs) which are in the frontage of new technology solutions for intelligent systems. However, real-time fault detection/diagnosis in such UAVs remains a challenge from data collection to prediction tasks. This work presents application of neural networks to detect in real-time elevon positioning faults. We show the efficiency of a formal method based on the Lipschitz constant for quantifying the stability of neural network models. We also present how this method can be coupled with spectral normalization constraints at the design phase to control the internal parameters of the model and make it more stable while keeping a high level of performance (accuracy-stability trade-off).

show abstract

Deep Neural Network Structures Solving Variational Inequalities

Cited by 94 publications

References 39 publications

Deep unfolding of a proximal interior point method for image restoration

Deep unfolding of a proximal interior point method for image restoration

Variational Models for Signal Processing with Graph Neural Networks

Safe Design of Stable Neural Networks for Fault Detection in Small UAVs

Contact Info

Product

Resources

About