Matrix Concentration Inequalities via the Method of Exchangeable Pairs

Mackey, Lester; Jordan, Michael I.; Chen, Richard Y.; Farrell, Brendan; Tropp, Joel A.

doi:10.21236/ada563088

Cited by 24 publications

(48 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We then arrive at upper bounds on P k X k > t and E k X k with the same forms as those of the proposed results for matrix i.d. series except that the term (10), (17) and (11)]. These results can also be regarded as an extension of the existing vector-version results (cf.…”

Section: Remark 32supporting

confidence: 60%

“…The following alternative expressions for the Bernstein-type result given in (10) and the H c -based result given in (17) can respectively be obtained: with probability at least 1 − δ,…”

Section: Remark 33mentioning

confidence: 99%

“…There have also been many other works on the eigenproblems of random matrices (cf. [16,17,18,19,20]), and the list provided here is incomplete.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Matrix Infinitely Divisible Series: Tail Inequalities and Their Applications

Zhang

Gao

Hsieh

et al. 2020

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

In this paper, we study tail inequalities of the largest eigenvalue of a matrix infinitely divisible (i.d.) series, which is a finite sum of fixed matrices weighted by i.d. random variables. We obtain several types of tail inequalities, including Bennett-type and Bernstein-type inequalities. This allows us to further bound the expectation of the spectral norm of a matrix i.d. series. Moreover, by developing a new lower-bound function for Q(s) = (s + 1) log(s + 1) − s that appears in the Bennett-type inequality, we derive a tighter tail inequality of the largest eigenvalue of the matrix i.d. series than the Bernstein-type inequality when the matrix dimension is high. The resulting lower-bound function is of independent interest and can improve any Bennett-type concentration inequality that involves the function Q(s). The class of i.d. probability distributions is large and includes Gaussian and Poisson distributions, among many others. Therefore, our results encompass the existing work [1] on matrix Gaussian series as a special case. Lastly, we show that the tail inequalities of a matrix i.d. series have applications in several optimization problems including the chance constrained optimization problem and the quadratic optimization problem with orthogonality constraints.

show abstract

Section: Remark 32supporting

confidence: 60%

“…The following alternative expressions for the Bernstein-type result given in (10) and the H c -based result given in (17) can respectively be obtained: with probability at least 1 − δ,…”

Section: Remark 33mentioning

confidence: 99%

See 1 more Smart Citation

Matrix Infinitely Divisible Series: Tail Inequalities and Their Applications

Zhang

Gao

Hsieh

et al. 2020

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

show abstract

“…Overall, for Lemma 6(i), it suffices to have spectral control over the covariance. In the special case where Y =X, we will accomplish this with the help of Matrix Hoeffding [14]. Before doing so, we consider Lemma 6(ii):…”

Section: Proof Of Theoremmentioning

confidence: 99%

“…Proposition 12 (Matrix Hoeffding [14]). Suppose {X j } j∈[n] are independent copies of a random symmetric matrix X ∈ R d×d such that EX = 0 and X 2→2 ≤ b almost surely.…”

Section: Proof Of Theoremmentioning

confidence: 99%

Matching Component Analysis for Transfer Learning

Clum¹,

Mixon²,

Scarnati³

2020

SIAM Journal on Mathematics of Data Science

View full text Add to dashboard Cite

We introduce a new Procrustes-type method called matching component analysis to isolate components in data for transfer learning. Our theoretical results describe the sample complexity of this method, and we demonstrate through numerical experiments that our approach is indeed well suited for transfer learning. IntroductionMany state-of-the-art classification algorithms require a large training set that is statistically similar to the test set. For example, deep learning-based approaches require a large number of representative samples in order to find near-optimal network weights and biases [4,9]. Similarly, template-based approaches require large dictionaries of training images so that each test image can be represented by an element of the dictionary [21,26,17,6]. For each technique, if test images cannot be represented in a feature space that has been determined from the training set, then classification accuracy is poor.In applications such as synthetic aperture radar (SAR) automatic target recognition (ATR), it is infeasible to collect the volume of data necessary to naively train high-accuracy classification networks. Additionally, due to varying operating conditions, the features measured in SAR imagery are different from those extracted from electro-optical (EO) imagery [13]. As such, off-the-shelf networks that have been pre-trained on the popular EObased ImageNet [3] or CIFAR-10 [8] datasets are insufficient for performing accurate ATR tasks in different imaging domains. In fact, recent work has demonstrated that pre-trained networks fail to effectively generalize to random perturbations on test sets [19,18]. To build more representative training sets, additional data are often generated using modeling and simulation software. However, due to various model errors, simulated data often misrepresent the real-world scattering observed in measured imagery. Thus, even though it is possible to augment training sets with a large amount of simulated data, the inherent differences in sensor modalities and data representations make modifying classification networks a non-trivial task [20].In this paper, we introduce matching component analysis (MCA) to help remedy this situation. Given a small number of images from the training domain and matching *

show abstract

Advanced Tools from Probability Theory

Foucart

Rauhut

2013

A Mathematical Introduction to Compressive Sensing

View full text Add to dashboard Cite

Matrix Concentration Inequalities via the Method of Exchangeable Pairs

Cited by 24 publications

References 17 publications

Matrix Infinitely Divisible Series: Tail Inequalities and Their Applications

Matrix Infinitely Divisible Series: Tail Inequalities and Their Applications

Matching Component Analysis for Transfer Learning

Advanced Tools from Probability Theory

Contact Info

Product

Resources

About