Diagonalization of tensors with circulant structure

Rezghi, Mansoor; Eldén, Lars

doi:10.1016/j.laa.2010.03.032

Cited by 32 publications

(17 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Different from a conventional convolutional layer, the circulant convolutional layer has a weight tensor W that exhibits circulant structure. In other words, the W of a circulant convolution layer is a 4D circulant tensors [19]. In general, a circulant tensor can exhibit circulant structure along any pair of its dimensions.…”

Section: Imposing Circulant Structure To Convolutional Layers Circulamentioning

confidence: 99%

“…3 shows that the weight tensor W of a circulant convolutional layer exhibits the circulant structure and has the reduced number of independent parameters. Besides, according to the tensor theory [19], circulant tensor also has the advantage of fast multiplication. Since multiplication is the kernel computation in neural network training and inference, the existence of fast multiplication of circulant tensor enables the immediate reduction in computational cost.…”

Section: Fast Forward and Backward Propagation Schemes On Circulant Cmentioning

confidence: 99%

“…, S}. According to [19,18], Fast Fourier Transform (FFT) can be used to accelerate the multiplication of a fiber and a slice of circulant tensor with time complexity reduced from O(N 2 ) to O(N log N ). Therefore, when W is a circulant tensor, Eq.…”

Section: Fast Forward and Backward Propagation Schemes On Circulant Cmentioning

confidence: 99%

See 2 more Smart Citations

CircConv: A Structured Convolution with Low Complexity

Liao

Yuan

2019

AAAI

View full text Add to dashboard Cite

Deep neural networks (DNNs), especially deep convolutional neural networks (CNNs), have emerged as the powerful technique in various machine learning applications. However, the large model sizes of DNNs yield high demands on computation resource and weight storage, thereby limiting the practical deployment of DNNs. To overcome these limitations, this paper proposes to impose the circulant structure to the construction of convolutional layers, and hence leads to circulant convolutional layers (CircConvs) and circulant CNNs. The circulant structure and models can be either trained from scratch or re-trained from a pre-trained non-circulant model, thereby making it very flexible for different training environments. Through extensive experiments, such strong structure-imposing approach is proved to be able to substantially reduce the number of parameters of convolutional layers and enable significant saving of computational cost by using fast multiplication of the circulant tensor.

show abstract

Section: Imposing Circulant Structure To Convolutional Layers Circulamentioning

confidence: 99%

Section: Fast Forward and Backward Propagation Schemes On Circulant Cmentioning

confidence: 99%

See 1 more Smart Citation

CircConv: A Structured Convolution with Low Complexity

Liao

Yuan

2019

AAAI

View full text Add to dashboard Cite

show abstract

“…Because of the circulant structure, we only require m n entries to construct a circulant tensor. This kind of circulant tensor has been studied in [25]. Any n × n circulant matrix C n can always be diagonalized by discrete Fourier matrices F n , [22] where…”

Section: Theorem 32 Let F Be a 2π -Periodic Real Continuous Functionmentioning

confidence: 99%

An eigenvalue problem for even order tensors with its applications

Cui

Chen

et al. 2015

Linear and Multilinear Algebra

View full text Add to dashboard Cite

In this paper, we study an eigenvalue problem for even order tensors. Using the matrix unfolding of even order tensors, we can establish the relationship between a tensor eigenvalue problem and a multilevel matrix eigenvalue problem. By considering a higher order singular value decomposition of a tensor, we show that higher order singular values are the square root of the eigenvalues of the product of the tensor and its conjugate transpose. This result is similar to that in matrix case. Also we study an eigenvalue problem for Toeplitz/circulant tensors, and give the lower and upper bounds of eigenvalues of Toeplitz tensors. An application in image restoration is also discussed.

show abstract

“…with ordered diagonal elements and (λ 6 In the terminology of [24] the operator is {1, 3}{2, 4}-diagonal; cf. (5.15).…”

Section: Perturbation Theory-general Casementioning

confidence: 99%

Perturbation Theory and Optimality Conditions for the Best Multilinear Rank Approximation of a Tensor

Eldén¹,

Savas²

2011

SIAM J. Matrix Anal. & Appl.

Self Cite

View full text Add to dashboard Cite

Abstract. The problem of computing the best rank-(p, q, r) approximation of a third order tensor is considered. First the problem is reformulated as a maximization problem on a product of three Grassmann manifolds. Then expressions for the gradient and the Hessian are derived in a local coordinate system at a stationary point, and conditions for a local maximum are given. A first order perturbation analysis is performed using the Grassmann manifold framework. The analysis is illustrated in a few examples, and it is shown that the perturbation theory for the singular value decomposition is a special case of the tensor theory.

show abstract

Diagonalization of tensors with circulant structure

Cited by 32 publications

References 15 publications

CircConv: A Structured Convolution with Low Complexity

CircConv: A Structured Convolution with Low Complexity

An eigenvalue problem for even order tensors with its applications

Perturbation Theory and Optimality Conditions for the Best Multilinear Rank Approximation of a Tensor

Contact Info

Product

Resources

About