Multiview Shared Subspace Learning Across Speakers and Speech Commands

Abstract

In many speech processing applications, the objective is to model different modes of variability to obtain robust speech features. In this paper, we learn speech representations in a multiview paradigm by constraining the views to known modes of variability such as speakers or spoken words. We use deep multiset canonical correlation (dMCCA) because it can model more than two views in parallel to learn a shared subspace across them. In order to model thousands of views (e.g., speakers), we demonstrate that stochastically sampling a small number of views generalizes dMCCA to the larger set of views. To evaluate our approach, we study two different aspects of the Speech Commands Dataset: variability among the speakers and speech commands. We show that, by treating observations from one mode of variability as multiple parallel views, we can learn representations that are discriminative to the other mode. We first consider different speakers as views of the same word to learn their shared subspace to represent an utterance. We then constrain the different words spoken by the same person as multiple views to learn speaker representations. Using classification and unsupervised clustering, we evaluate the efficacy of multiview representations to identify speech commands and speakers.

CAST: Character labeling in Animation using Self-supervision by Tracking

“…Schroff et al [19] have introduced FaceNet and the triplet-loss for projecting images onto a latent space that quantifies similarity in a supervised-learning manner. Recently, Somandepalli et al [20] used tracking of faces in a photo-realistic video, followed by clustering and verification using MvCorr [21] and Improved Triplet [22] to adapt available face representation data to perform better on racially diverse images following [23]. Aneja et al [24] have suggested DeepExpr model for facial expression recognition for multiple styles.…”

Generalized Multiview Shared Subspace Learning Using View Bootstrapping

“…We first review the multi-view correlation (mv-corr) objective developed by Somandepalli et. al., (2019a;2019b).…”

“…recordings from over 1800 speakers saying one or more of 30 commands such as "On" and "Off". The application of mv-corr for spoken-word recognition and text-dependent speaker recognition in SCD was studied by Somandepalli et al (2019a) for speaker recognition task compared to the SoA in this domain (Snyder et al, 2017). Building upon their work, in this paper, we analyze spoken-word recognition on SCD in a greater detail.…”

“…Speaker verification methods based on joint factor analysis (Dehak et al, 2009) and total variability modeling (Dehak et al, 2011) have explored the ideas of factoring out the speaker-dependent factors and speaker-independent factors to obtain robust speaker representations in the context of domain adaption. Recently, Somandepalli et al (2019a) showed that a more robust speech representation can be obtained by explicitly modeling multiple utterances of a word as corresponding views.…”

“…In this paper, we build upon the work by Somandepalli et. al., (2019a;2019b) where a multi-view correlation objective (mv-corr) was proposed to learn shared subspaces across multiple views.…”