The Optimal Hard Threshold for Singular Values is <inline-formula> <tex-math notation="TeX">\(4/\sqrt {3}\) </tex-math></inline-formula>

Abstract

We consider recovery of low-rank matrices from noisy data by hard thresholding of singular values, in which empirical singular values below a threshold λ are set to 0. We study the asymptotic mean squared error (AMSE) in a framework, where the matrix size is large compared with the rank of the matrix to be recovered, and the signal-to-noise ratio of the low-rank piece stays constant. The AMSE-optimal choice of hard threshold, in the case of n-by-n matrix in white noise of level σ , is simply (4/ √ 3) √ nσ ≈ 2.309 √ nσ when σ is known, or simply 2.858 · y med when σ is unknown, where y med is the median empirical singular value. For nonsquare, m by n matrices with m = n the thresholding coefficients 4/ √ 3 and 2.858 are replaced with different provided constants that depend on m/n. Asymptotically, this thresholding rule adapts to unknown rank and unknown noise level in an optimal manner: it is always better than hard thresholding at any other value, and is always better than ideal truncated singular value decomposition (TSVD), which truncates at the true rank of the low-rank matrix we are trying to recover. Hard thresholding at the recommended value to recover an n-by-n matrix of rank r guarantees an AMSE at most 3 nrσ 2 . In comparison, the guarantees provided by TSVD, optimally tuned singular value soft thresholding and the best guarantee achievable by any shrinkage of the data singular values are 5 nrσ 2 , 6 nrσ 2 , and 2 nrσ 2 , respectively. The recommended value for hard threshold also offers, among hard thresholds, the best possible AMSE guarantees for recovering matrices with bounded nuclear norm. Empirical evidence suggests that performance improvement over TSVD and other popular shrinkage rules can be substantial, for different noise distributions, even in relatively small n.Index Terms-Singular values shrinkage, optimal threshold, low-rank matrix denoising, unique admissible, scree plot elbow truncation, quarter circle law, bulk edge. 0018-9448

Development and validation of warning system of ventricular tachyarrhythmia in patients with heart failure with heart rate variability data

“…Here, we used the method of Gavish and Donoho to determine the optimal number of principal components for the purpose of training the classifiers and prediction. [27]…”

Compressed dynamic mode decomposition for background modeling

“…One alternative to the predefined target-rank k is the recent hard-thresholding algorithm of Gavish and Donoho [25]. This method can can be combined with step 4 to automatically determine the optimal target-rank.…”

“…While, we use here a fixed number of samples, the choice can be guided by the formula p > k · log(n/k). The target-rank k is automatically determined via the optimal hard-threshold for singular values [25]. Once the dynamic mode decomposition is obtained, the optimal set of modes is selected using the orthogonal matching pursuit method.…”

Including inputs and control within equation-free architectures for complex systems

“…The theorem states that the best approximation of X with k modes can be found by retaining the k largest singular values and respective modes. Recent theoretical developments attempt to optimally identify r when X may have additive noise [82,83]. …”