Image classification using local tensor singular value decompositions

Newman, Elizabeth; Kilmer, Misha E.; Horesh, Lior

doi:10.1109/camsap.2017.8313137

Cited by 19 publications

(16 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Application areas where the t-product has proved useful include facial recognition, 14 tomographic image reconstruction, 15 video completion, 16 and image classification. 17 Several matrix-based methods, that do not apply the t-product, for solving problems of the form (1) have recently been described in the literature. Beik et al 11 introduced the GKT-based tensor format (GKT_BTF) method and applied it to determine the solution of large-scale ill-posed Sylvester and Stein equations.…”

Section: Related Prior Work and Some Applicationsmentioning

confidence: 99%

See 2 more Smart Citations

The tensor Golub–Kahan–Tikhonov method applied to the solution of ill‐posed problems with a t‐product structure

Reichel

Ugwu

2021

Numerical Linear Algebra App

View full text Add to dashboard Cite

This paper discusses an application of partial tensor Golub-Kahan bidiagonalization to the solution of large-scale linear discrete ill-posed problems based on the t-product formalism for third-order tensors proposed by Kilmer and Martin (M. E. Kilmer and C. D. Martin, Factorization strategies for third order tensors, Linear Algebra Appl., 435 (2011), pp. 641-658). The solution methods presented first reduce a given (large-scale) problem to a problem of small size by application of a few steps of tensor Golub-Kahan bidiagonalization and then regularize the reduced problem by Tikhonov's method. The regularization operator is a third-order tensor, and the data may be represented by a matrix, that is, a tensor slice, or by a general third-order tensor. A regularization parameter is determined by the discrepancy principle. This results in fully automatic solution methods that neither require a user to choose the number of bidiagonalization steps nor the regularization parameter. The methods presented extend available methods for the solution for linear discrete ill-posed problems defined by a matrix operator to linear discrete ill-posed problems defined by a third-order tensor operator. An interlacing property of singular tubes for third-order tensors is shown and applied. Several algorithms are presented. Computed examples illustrate the advantage of the tensor t-product approach, in comparison with solution methods that are based on matricization of the tensor equation. K E Y W O R D Sdiscrepancy principle, discrete ill-posed problem, t-product, tensor Golub-Kahan bidiagonalization, Tikhonov regularization 1 where  = [a ijk ] 𝓁,m,n i,j,k=1 ∈ R 𝓁×m×n is a third-order tensor, ⃗  ∈ R m×1×n and ⃗  ∈ R 𝓁×1×n are lateral slices of third-order tensors and may be thought of as laterally oriented matrices. The operation * denotes the tensor t-product introduced in the seminal work by Kilmer and Martin 1 and applied to image deblurring problems by Kilmer et al. 1,2 The t-product between

show abstract

Section: Related Prior Work and Some Applicationsmentioning

confidence: 99%

“…Following Newman et al, 17 we define the range of the tensor  as the t-linear span of the lateral slices of ,…”

Section: Notation and Preliminariesmentioning

confidence: 99%

See 1 more Smart Citation

The tensor Golub–Kahan–Tikhonov method applied to the solution of ill‐posed problems with a t‐product structure

Reichel

Ugwu

2021

Numerical Linear Algebra App

View full text Add to dashboard Cite

show abstract

“…We focus on supervised classification problems where data are high order tensors. Kernel-based classification methods [10,11] require a similarity measure. It is standard [9] to consider a kernel between two tensors X and Y as for instance the RBF Gaussian kernel :…”

Section: Kernel For Data Tensorsmentioning

confidence: 99%

Speeding Up of Kernel-Based Learning for High-Order Tensors

Karmouda

Boulanger

Boyer

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Supervised learning is a major task to classify datasets. In our context, we are interested into classification from highorder tensors datasets. The "curse of dimensionality" states that the complexities in terms of storage and computation grow exponentially with the order. As a consequence, the method from the state-of-art based on the Higher-Order SVD (HOSVD) works well but suffers from severe limitation in terms of complexities. In this work, we propose a fast Grassmannian kernel-based method for high-order tensor learning based on the equivalence between the Tucker and the tensortrain decompositions. Our solution is linked to the tensor network, where the aim is to break the initial high-order tensor into a collection of low-order tensors (at most 3-order). We show on several real datasets that the proposed method reaches a similar accuracy classification rate as the Grassmannian kernel-based method based on the HOSVD but for a much lower complexity.

show abstract

“…; IBM TJ Watson Research Center. § Tel Aviv University [34], video completion [36], and image classification [25]. In our context, the aforementioned fully-connected layers are replaced with layers of the form A j`1 " σpW j ˚Aj ` B j q where A j`1 , A j , W j are tensors, and product ˚is the t-product introduced in [16].…”

mentioning

confidence: 99%

Stable Tensor Neural Networks for Rapid Deep Learning

Newman,

Horesh,

Avron

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

We propose a tensor neural network (t-NN) framework that offers an exciting new paradigm for designing neural networks with multidimensional (tensor) data. Our network architecture is based on the t-product [16], an algebraic formulation to multiply tensors via circulant convolution. In this t-product algebra, we interpret tensors as t-linear operators analogous to matrices as linear operators, and hence our framework inherits mimetic matrix properties. To exemplify the elegant, matrix-mimetic algebraic structure of our t-NNs, we expand on recent work [8] which interprets deep neural networks as discretizations of non-linear differential equations and introduces stable neural networks which promote superior generalization. Motivated by this dynamic framework, we introduce a stable t-NN which facilitates more rapid learning because of its reduced, more powerful parameterization. Through our high-dimensional design, we create a more compact parameter space and extract multidimensional correlations otherwise latent in traditional algorithms. We further generalize our t-NN framework to a family of tensor-tensor products [14] which still induce a matrix-mimetic algebraic structure. Through numerical experiments on the MNIST [20] and CIFAR-10 [17] datasets, we demonstrate the more powerful parameterizations and improved generalizability of stable t-NNs.

show abstract

Image classification using local tensor singular value decompositions

Cited by 19 publications

References 15 publications

The tensor Golub–Kahan–Tikhonov method applied to the solution of ill‐posed problems with a t‐product structure

The tensor Golub–Kahan–Tikhonov method applied to the solution of ill‐posed problems with a t‐product structure

Speeding Up of Kernel-Based Learning for High-Order Tensors

Stable Tensor Neural Networks for Rapid Deep Learning

Contact Info

Product

Resources

About