Gaussian bandwidth selection for manifold learning and classification

Lindenbaum, Ofir; Salhov, Moshe; Yeredor, Arie; Averbuch, Amir

doi:10.1007/s10618-020-00692-x

Cited by 20 publications

(10 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2 [46], examine several other values around that median, and then choose the value yielding the best result. For more sophisticated approach to tune the Gaussian bandwidth, see [24]. In this example, we used σ F = 0.007 and σ F = 0.07 after several trials starting from the simple median approach in order to demonstrate the effect of this parameter for the eGHWT best basis.…”

Section: Another Way To Construct a Graph From An Image For Efficient Approximationmentioning

confidence: 99%

eGHWT: The Extended Generalized Haar–Walsh Transform

Saito

Shao

2022

J Math Imaging Vis

View full text Add to dashboard Cite

Extending computational harmonic analysis tools from the classical setting of regular lattices to the more general setting of graphs and networks is very important, and much research has been done recently. The generalized Haar–Walsh transform (GHWT) developed by Irion and Saito (2014) is a multiscale transform for signals on graphs, which is a generalization of the classical Haar and Walsh–Hadamard transforms. We propose the extended generalized Haar–Walsh transform (eGHWT), which is a generalization of the adapted time–frequency tilings of Thiele and Villemoes (1996). The eGHWT examines not only the efficiency of graph-domain partitions but also that of “sequency-domain” partitions simultaneously. Consequently, the eGHWT and its associated best-basis selection algorithm for graph signals significantly improve the performance of the previous GHWT with the similar computational cost, $$O(N \log N)$$ O ( N log N ) , where N is the number of nodes of an input graph. While the GHWT best-basis algorithm seeks the most suitable orthonormal basis for a given task among more than $$(1.5)^N$$ ( 1.5 ) N possible orthonormal bases in $$\mathbb {R}^N$$ R N , the eGHWT best-basis algorithm can find a better one by searching through more than $$0.618\cdot (1.84)^N$$ 0.618 · ( 1.84 ) N possible orthonormal bases in $$\mathbb {R}^N$$ R N . This article describes the details of the eGHWT best-basis algorithm and demonstrates its superiority using several examples including genuine graph signals as well as conventional digital images viewed as graph signals. Furthermore, we also show how the eGHWT can be extended to 2D signals and matrix-form data by viewing them as a tensor product of graphs generated from their columns and rows and demonstrate its effectiveness on applications such as image approximation.

show abstract

Section: Another Way To Construct a Graph From An Image For Efficient Approximationmentioning

confidence: 99%

eGHWT: The Extended Generalized Haar–Walsh Transform

Saito

Shao

2022

J Math Imaging Vis

View full text Add to dashboard Cite

show abstract

“…We note that one can use the alternative and popular Gaussian affinity, i.e., exp(−d(φ i−1 , φ j−1 ) 2 / ). This affinity, however, requires a user to select an appropriate scale parameter > 0, which is not a trivial task as explained in [33], for example. Moreover, our edge weights using the inverse distances tend to connect the eigenvectors more globally compared to the Gaussian affinity with a fixed bandwidth.…”

Section: Natural Graph Wavelet Packets Using Varimax Rotationsmentioning

confidence: 99%

Natural Graph Wavelet Packet Dictionaries

Cloninger

Saito

2021

J Fourier Anal Appl

View full text Add to dashboard Cite

We introduce a set of novel multiscale basis transforms for signals on graphs that utilize their “dual” domains by incorporating the “natural” distances between graph Laplacian eigenvectors, rather than simply using the eigenvalue ordering. These basis dictionaries can be seen as generalizations of the classical Shannon wavelet packet dictionary to arbitrary graphs, and do not rely on the frequency interpretation of Laplacian eigenvalues. We describe the algorithms (involving either vector rotations or orthogonalizations) to construct these basis dictionaries, use them to efficiently approximate graph signals through the best basis search, and demonstrate the strengths of these basis dictionaries for graph signals measured on sunflower graphs and street networks.

show abstract

“…This global scale guarantees that each point is connected to at least one other point. Alternatively, adaptive scales could be used as suggested in [22]. We then compute P via…”

Section: Examplesmentioning

confidence: 99%

“…This global scale guarantees that each point is connected to at least one other point. Alternatively, adaptive scales could be used as suggested in [ 22 ]. We then compute P via

The matrix P can be interpreted as a random walk over the data points, (see for example [ 18 ]).…”

Section: Examplesmentioning

confidence: 99%

The Spectral Underpinning of word2vec

Jaffe

Kluger

Lindenbaum

et al. 2020

Front. Appl. Math. Stat.

Self Cite

View full text Add to dashboard Cite

Word2vec introduced by Mikolov et al. is a word embedding method that is widely used in natural language processing. Despite its success and frequent use, a strong theoretical justification is still lacking. The main contribution of our paper is to propose a rigorous analysis of the highly nonlinear functional of word2vec. Our results suggest that word2vec may be primarily driven by an underlying spectral method. This insight may open the door to obtaining provable guarantees for word2vec. We support these findings by numerical simulations. One fascinating open question is whether the nonlinear properties of word2vec that are not captured by the spectral method are beneficial and, if so, by what mechanism.

show abstract

Gaussian bandwidth selection for manifold learning and classification

Cited by 20 publications

References 46 publications

eGHWT: The Extended Generalized Haar–Walsh Transform

eGHWT: The Extended Generalized Haar–Walsh Transform

Natural Graph Wavelet Packet Dictionaries

The Spectral Underpinning of word2vec

Contact Info

Product

Resources

About