Closed Form Word Embedding Alignment

Dev, Sunipa; Hassan, Safia; Phillips, Jeff M.

doi:10.1109/icdm.2019.00023

Cited by 5 publications

(5 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multiple fitting iterations result in sets of vectors with similar relative positions among each other, i.e. similar cosine angles between node pairs, but they generally do not retain their absolute values ( Dev et al, 2019 ; for the stability of the relations among embeddings see Wang et al, 2020 ). As a result, utilizing the CE framework to explore individual differences among subjects requires a method which would align different CEs to the same latent space ( Fig.…”

Section: Resultsmentioning

confidence: 99%

“…Independent fitting iterations of the node2vec algorithm resulted in sets of vectors with similar cosine angle between each node pairs, but not necessarily similar absolute values ( Dev et al, 2019 ). Here we demonstrate our novel approach which enables us to align separately learned CE to the same latent space (see Fig.…”

Section: Methodsmentioning

confidence: 99%

“…However, utilizing these nodal representations for predicting function and behavior requires their mutual alignment to the same latent space to allow comparison across individuals. Multiple fitting iterations of the Word2Vec algorithm, even on the same brain graph, result in sets of vectors with similar relative positions among each other but with different absolute values ( Dev et al, 2019 ; for the stability of the relations among embeddings see Wang et al, 2020 ). In the domain of machine translation, this is typically addressed by finding a transformation that minimizes word distances from one language to another ( Smith et al, 2017 ).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Mapping individual differences across brain network structure to function and behavior with connectome embedding

et al. 2021

View full text Add to dashboard Cite

The connectome, a comprehensive map of the brain’s anatomical connections, is often summarized as a matrix comprising all dyadic connections among pairs of brain regions. This representation cannot capture higher-order relations within the brain graph. Connectome embedding (CE) addresses this limitation by creating compact vectorized representations of brain nodes capturing their context in the global network topology. Here, nodes “context” is defined as random walks on the brain graph and as such, represents a generative model of diffusive communication around nodes. Applied to group-averaged structural connectivity, CE was previously shown to capture relations between inter-hemispheric homologous brain regions and uncover putative missing edges from the network reconstruction. Here we extend this framework to explore individual differences with a novel embedding alignment approach. We test this approach in two lifespan datasets (NKI: n = 542; Cam-CAN: n = 601) that include diffusion-weighted imaging, resting-state fMRI, demographics and behavioral measures. We demonstrate that modeling functional connectivity with CE substantially improves structural to functional connectivity mapping both at the group and subject level. Furthermore, age-related differences in this structure-function mapping, are preserved and enhanced. Importantly, CE captures individual differences by out-of-sample prediction of age and intelligence. The resulting predictive accuracy was higher compared to using structural connectivity and functional connectivity. We attribute these findings to the capacity of the CE to incorporate aspects of both anatomy (the structural graph) and function (diffusive communication). Our novel approach allows mapping individual differences in the connectome through structure to function and behavior.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Mapping individual differences across brain network structure to function and behavior with connectome embedding

et al. 2021

View full text Add to dashboard Cite

show abstract

“…However, if these representations can be used for LR understanding of the represented features, properties such as the relative positions of word-embeddings used to complete analogies should be evident in the respective representations despite these differences. Dev et al (2019) explain that "rotation or scaling of the entire dataset will not affect synonyms (nearest neighbors), linear substructures (dot products), analogies, or linear classifiers" because "there is nothing extrinsic about any of these properties." For example, in studying the impact of basis rotations to align GloVe (Pennington et al 2014) and Word2Vec (Mikolov et al 2013) embeddings, they confirm that using a vector from Word2Vec to complete an analogy using GloVe embeddings "is very poor, close to 0; that is, extrinsically there is very little information carried over" by the (basis dependent) parameter values themselves.…”

Section: The Target Of ML Modelsmentioning

confidence: 99%

Understanding from Deep Learning Models in Context

Tamir¹,

Shech²

2022

Scientific Understanding and Representation

View full text Add to dashboard Cite

This paper places into context how the term model in machine learning (ML) contrasts with traditional usages of scientific models for understanding and we show how direct analysis of an estimator's learned transformations (specifically, the hidden layers of a deep learning model) can improve understanding of the target phenomenon and reveal how the model organizes relevant information. Specifically, three modes of understanding will be identified, the difference between implementation irrelevance and functionally approximate irrelevance will be disambiguated, and how this distinction impacts potential understanding with these models will be explored.Additionally, by distinguishing between empirical link failures from representational ones, an ambiguity in the concept of link uncertainty will be addressed thus clarifying the role played by scientific background knowledge in enabling understanding with ML.

show abstract

“…6 ) between them. DSMs are first aligned using absolute orientation with scaling (see Algorithm 1 below from Dev et al, 2018 , originally Algorithm 2.4 in their paper) where the optimal alignment is obtained by minimizing the sum of squared errors under the Euclidian distance between all pairs of common data points, using linear transformations—rotation and scaling—which do not alter inner cosine similarity metrics and hence preserve measures of pairwise lexical similarity.…”

Section: Model and Experimental Setupmentioning

confidence: 99%

Avoiding Conflict: When Speaker Coordination Does Not Require Conceptual Agreement

Kabbach

Herbelot

2021

Front. Artif. Intell.

View full text Add to dashboard Cite

In this paper we discuss the socialization hypothesis—the idea that speakers of the same (linguistic) community should share similar concepts given that they are exposed to similar environments and operate in highly-coordinated social contexts—and challenge the fact that it is assumed to constitute a prerequisite to successful communication. We do so using distributional semantic models of meaning (DSMs) which create lexical representations via latent aggregation of co-occurrence information between words and contexts. We argue that DSMs constitute particularly adequate tools for exploring the socialization hypothesis given that 1) they provide full control over the notion of background environment, formally characterized as the training corpus from which distributional information is aggregated; and 2) their geometric structure allows for exploiting alignment-based similarity metrics to measure inter-subject alignment over an entire semantic space, rather than a set of limited entries. We propose to model coordination between two different DSMs trained on two distinct corpora as dimensionality selection over a dense matrix obtained via Singular Value Decomposition This approximates an ad-hoc coordination scenario between two speakers as the attempt to align their similarity ratings on a set of word pairs. Our results underline the specific way in which linguistic information is spread across singular vectors, and highlight the need to distinguish agreement from mere compatibility in alignment-based notions of conceptual similarity. Indeed, we show that compatibility emerges from idiosyncrasy so that the unique and distinctive aspects of speakers’ background experiences can actually facilitate—rather than impede—coordination and communication between them. We conclude that the socialization hypothesis may constitute an unnecessary prerequisite to successful communication and that, all things considered, communication is probably best formalized as the cooperative act of avoiding conflict, rather than maximizing agreement.

show abstract

Closed Form Word Embedding Alignment

Cited by 5 publications

References 26 publications

Mapping individual differences across brain network structure to function and behavior with connectome embedding

Mapping individual differences across brain network structure to function and behavior with connectome embedding

Understanding from Deep Learning Models in Context

Avoiding Conflict: When Speaker Coordination Does Not Require Conceptual Agreement

Contact Info

Product

Resources

About