Parseval Proximal Neural Networks

Hasannasab, Marzieh; Hertrich, Johannes; Neumayer, Sebastian; Plonka, Gerlind; Setzer, Simon; Steidl, Gabriele

doi:10.1007/s00041-020-09761-7

Cited by 44 publications

(35 citation statements)

References 33 publications

(51 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The unfolded networks we consider here fall into the larger class of proximal neural networks studied in [17][18][19]. Many other related works are in the context of dictionary learning or sparse coding: The central problem of sparse coding is to learn weight matrices for an unfolded version of ISTA.…”

Section: Related Workmentioning

confidence: 99%

Generalization Error Bounds for Iterative Recovery Algorithms Unfolded as Neural Networks

Schnoor¹,

Behboodi²,

Rauhut³

2021

Preprint

View full text Add to dashboard Cite

Motivated by the learned iterative soft thresholding algorithm (LISTA), we introduce a general class of neural networks suitable for sparse reconstruction from few linear measurements. By allowing a wide range of degrees of weight-sharing between the layers, we enable a unified analysis for very different neural network types, ranging from recurrent ones to networks more similar to standard feedforward neural networks. Based on training samples, via empirical risk minimization we aim at learning the optimal network parameters and thereby the optimal network that reconstructs signals from their low-dimensional linear measurements. We derive generalization bounds by analyzing the Rademacher complexity of hypothesis classes consisting of such deep networks, that also take into account the thresholding parameters. We obtain estimates of the sample complexity that essentially depend only linearly on the number of parameters and on the depth. We apply our main result to obtain specific generalization bounds for several practical examples, including different algorithms for (implicit) dictionary learning, and convolutional neural networks.

show abstract

Section: Related Workmentioning

confidence: 99%

Generalization Error Bounds for Iterative Recovery Algorithms Unfolded as Neural Networks

Schnoor¹,

Behboodi²,

Rauhut³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently, neural networks on infinite dimensional spaces have received a certain attention [23]. For applications of their finite dimensional counterparts, we refer to [29,30]. To this end, let (X, • ) be a real p -space, p ∈ (1, ∞) and A k : X → X, k = 1, 2, .…”

Section: Deep Learningmentioning

confidence: 99%

On $$\alpha $$-Firmly Nonexpansive Operators in r-Uniformly Convex Spaces

Бeрделлима

Steidl

2021

Results Math

Self Cite

View full text Add to dashboard Cite

We introduce the class of $$\alpha $$ α -firmly nonexpansive and quasi $$\alpha $$ α -firmly nonexpansive operators on r-uniformly convex Banach spaces. This extends the existing notion from Hilbert spaces, where $$\alpha $$ α -firmly nonexpansive operators coincide with so-called $$\alpha $$ α -averaged operators. For our more general setting, we show that $$\alpha $$ α -averaged operators form a subset of $$\alpha $$ α -firmly nonexpansive operators. We develop some basic calculus rules for (quasi) $$\alpha $$ α -firmly nonexpansive operators. In particular, we show that their compositions and convex combinations are again (quasi) $$\alpha $$ α -firmly nonexpansive. Moreover, we will see that quasi $$\alpha $$ α -firmly nonexpansive operators enjoy the asymptotic regularity property. Then, based on Browder’s demiclosedness principle, we prove for r-uniformly convex Banach spaces that the weak cluster points of the iterates $$x_{n+1}:=Tx_{n}$$ x n + 1 : = T x n belong to the fixed point set $${{\,\mathrm{Fix}\,}}T$$ Fix T whenever the operator T is nonexpansive and quasi $$\alpha $$ α -firmly. If additionally the space has a Fréchet differentiable norm or satisfies Opial’s property, then these iterates converge weakly to some element in $${{\,\mathrm{Fix}\,}}T$$ Fix T . Further, the projections $$P_{{{\,\mathrm{Fix}\,}}T}x_n$$ P Fix T x n converge strongly to this weak limit point. Finally, we give three illustrative examples, where our theory can be applied, namely from infinite dimensional neural networks, semigroup theory, and contractive projections in $$L_p$$ L p , $$p \in (1,\infty ) \backslash \{2\}$$ p ∈ ( 1 , ∞ ) \ { 2 } spaces on probability measure spaces.

show abstract

“…Recently, neural networks on infinite dimensional spaces have received a certain attention [21]. For applications of their finite dimensional counterparts, we refer to [27,28]. To this end, let (X, • ) be a real ℓ p -space, p ∈ (1, ∞) and A k : X → X, k = 1, 2, ..., d a family of affine mappings which are α k -firmly nonexpansive.…”

Section: Deep Learningmentioning

confidence: 99%

On $α$-Firmly Nonexpansive Operators in $r$-Uniformly Convex Spaces

Бeрделлима¹,

Steidl²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

We introduce the class of α-firmly nonexpansive and quasi α-firmly nonexpansive operators on r-uniformly convex Banach spaces. This extends the existing notion from Hilbert spaces, where α-firmly nonexpansive operators coincide with so-called α-averaged operators. For our more general setting, we show that α-averaged operators form a subset of α-firmly nonexpansive operators. We develop some basic calculus rules for (quasi) α-firmly nonexpansive operators. In particular, we show that their compositions and convex combinations are again (quasi) α-firmly nonexpansive. Moreover, we will see that quasi α-firmly nonexpansive operators enjoy the asymptotic regularity property. Then, based on Browder's demiclosedness principle, we prove for r-uniformly convex Banach spaces that the weak cluster points of the iterates x n+1 := T x n belong to the fixed point set Fix T whenever the operator T is nonexpansive and quasi α-firmly. If additionally the space has a Fréchet differentiable norm or satisfies Opial's property, then these iterates converge weakly to some element in Fix T . Further, the projections P Fix T x n converge strongly to this weak limit point. Finally, we give three illustrative examples, where our theory can be applied, namely from infinite dimensional neural networks, semigroup theory, and contractive projections in L p , p ∈ (1, ∞)\{2} spaces on probability measure spaces.

show abstract

Parseval Proximal Neural Networks

Cited by 44 publications

References 33 publications

Generalization Error Bounds for Iterative Recovery Algorithms Unfolded as Neural Networks

Generalization Error Bounds for Iterative Recovery Algorithms Unfolded as Neural Networks

On $$\alpha $$-Firmly Nonexpansive Operators in r-Uniformly Convex Spaces

On $α$-Firmly Nonexpansive Operators in $r$-Uniformly Convex Spaces

Contact Info

Product

Resources

About