MultiView Diffusion Maps

Lindenbaum, Ofir; Yeredor, Arie; Salhov, Moshe; Averbuch, Amir

doi:10.48550/arxiv.1508.05550

Cited by 11 publications

(21 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the multiple view case, the kernel bandwidth in each view is typically set in the literature as if the data is captured only in a single view and also require graph connectivity for each view, e.g, in [12], [16], [20], [21]. In contrast, we show that when the data is measured in multiple views, the graph of each single view does not necessarily have to be connected.…”

Section: Graph Theory Interpretation For Kernel Bandwidth Selectionmentioning

confidence: 86%

“…The potential of improving the robustness of the obtained representations to interferences by fusing data captured in multiple views, has recently motivated researchers extending kernel-based geometric methods to the multiple views case [10]- [21]. Among these studies, we mention the studies presented in [12], [16], [20], [21] sharing similar ideas of constructing separate affinity kernels for each view, and fusing the data by the product between the affinity kernels. A method of particular interest in this work was presented in [21], where special emphasis is given to the robustness of the fusion process to interferences.…”

Section: Introductionmentioning

confidence: 99%

“…We note that throughout this paper we consider the selection of the kernel bandwidth for the alternating diffusion maps method presented in [21]. However, the provided analysis and the proposed algorithm may be extended with mild modifications to the methods presented in [12], [16], [20], which are also based on the product between the affinity kernels of the views.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Kernel-Based Sensor Fusion With Application to Audio-Visual Voice Activity Detection

Dov

Talmon

Cohen

2016

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

Abstract-In this paper, we address the problem of multiple view data fusion in the presence of noise and interferences. Recent studies have approached this problem using kernel methods, by relying particularly on a product of kernels constructed separately for each view. From a graph theory point of view, we analyze this fusion approach in a discrete setting. More specifically, based on a statistical model for the connectivity between data points, we propose an algorithm for the selection of the kernel bandwidth, a parameter, which, as we show, has important implications on the robustness of this fusion approach to interferences. Then, we consider the fusion of audio-visual speech signals measured by a single microphone and by a video camera pointed to the face of the speaker. Specifically, we address the task of voice activity detection, i.e., the detection of speech and non-speech segments, in the presence of structured interferences such as keyboard taps and office noise. We propose an algorithm for voice activity detection based on the audio-visual signal. Simulation results show that the proposed algorithm outperforms competing fusion and voice activity detection approaches. In addition, we demonstrate that a proper selection of the kernel bandwidth indeed leads to improved performance.

show abstract

Section: Graph Theory Interpretation For Kernel Bandwidth Selectionmentioning

confidence: 86%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Kernel-Based Sensor Fusion With Application to Audio-Visual Voice Activity Detection

Dov

Talmon

Cohen

2016

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

show abstract

“…First, the heat kernel exists and is the unique positive function satisfying (H 1 ), (H 2 ), and (H 3 ). Additionally, the solution to the initial value problem, (16) ∂u/∂t = ∆ g(t) u, u(x, 0) = f (x), has the integral representation [24, Corollary 2.2]:…”

Section: The Heat Kernel For (M G(•))mentioning

confidence: 99%

“…The method is based on the product operators P 1 P 2 , and P 2 P 1 , where P 1 and P 2 are constructed from two different views of the data. In [16], Lindenbaum, Yeredor, Salhov, and Averbuch follow a similar approach, but concatenate a collection of alternating products in block matrix defining Multi-View Diffusion Maps. Recently, several applications and extensions of Alternating Diffusion have been developed.…”

Section: Introductionmentioning

confidence: 99%

Time coupled diffusion maps

Marshall

Hirn

2018

Applied and Computational Harmonic Analysis

View full text Add to dashboard Cite

Abstract. We consider a collection of n points in R d measured at m times, which are encoded in an n × d × m data tensor. Our objective is to define a single embedding of the n points into Euclidean space which summarizes the geometry as described by the data tensor. In the case of a fixed data set, diffusion maps and related graph Laplacian methods define such an embedding via the eigenfunctions of a diffusion operator constructed on the data. Given a sequence of m measurements of n points, we introduce the notion of time coupled diffusion maps which have natural geometric and probabilistic interpretations. To frame our method in the context of manifold learning, we model evolving data as samples from an underlying manifold with a time-dependent metric, and we describe a connection of our method to the heat equation on such a manifold.

show abstract

Latent common manifold learning with alternating diffusion: Analysis and applications

Talmon

2019

Applied and Computational Harmonic Analysis

View full text Add to dashboard Cite

The analysis of data sets arising from multiple sensors has drawn significant research attention over the years. Traditional methods, including kernel-based methods, are typically incapable of capturing nonlinear geometric structures. We introduce a latent common manifold model underlying multiple sensor observations for the purpose of multimodal data fusion. A method based on alternating diffusion is presented and analyzed; we provide theoretical analysis of the method under the latent common manifold model. To exemplify the power of the proposed framework, experimental results in several applications are reported.

show abstract

MultiView Diffusion Maps

Cited by 11 publications

References 0 publications

Kernel-Based Sensor Fusion With Application to Audio-Visual Voice Activity Detection

Kernel-Based Sensor Fusion With Application to Audio-Visual Voice Activity Detection

Time coupled diffusion maps

Latent common manifold learning with alternating diffusion: Analysis and applications

Contact Info

Product

Resources

About