How Well Sentence Embeddings Capture Meaning

White, Lyndon; Togneri, Roberto; Liu, Wei; Bennamoun, Mohammed

doi:10.1145/2838931.2838932

Cited by 45 publications

(38 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite the large number of such approaches, it is still not clear that the composed phrase or sentence embeddings express the intended meaning, as recently shown by Shwartz and Dagan (2019), Zhu et al (2018) and Dasgupta et al (2018). Even more interesting is the fact that averaging and weighted averaging approaches have been shown to outperform complex deep learning methods (White et al, 2015;Wieting et al, 2016;Arora et al, 2017). This shows potential in exploiting the merits of simpler approaches but boosting them up with more powerful intuitive and linguistic constraints, as the ones proposed in this work.…”

Section: Relevant Workmentioning

confidence: 99%

Section: Evaluation Of the Constraints 41 Compared Approachesmentioning

confidence: 99%

“…Baseline approaches We include baseline operations from the literature that were recently shown to outperform complex deep architectures (White et al, 2015;Wieting et al, 2016;Arora et al, 2017). We use weighted elementwise vector addition (1) and multiplication (2) (Mitchell and Lapata, 2010;Turney, 2012;White et al, 2015;Hartung et al, 2017;Arora et al, 2017) and weighted elementwise average (3) (Mikolov et al, 2013;Wieting et al, 2016). Since addition and multiplication have been shown to perform so strongly and since multiplicative models have the drawback that the presence of zeroes in either of the vectors leads to information essentially being lost, we follow Mitchell and Lapata (2010) and also include a fourth equation, combining the addition and multiplication operations (4).…”

Section: Evaluation Of the Constraints 41 Compared Approachesmentioning

confidence: 99%

“…Deep Learning approach Although White et al (2015), Wieting et al (2016) and Arora et al (2017) found that simple operations outperform complex deep architectures, there is still value in comparing the performance of a trained neural net to the performance of the other methods. For this purpose we experimented with multiple architectures, including feedforward nets, RNNs and LSTMs, attempting to find the best that fits our data.…”

Section: Evaluation Of the Constraints 41 Compared Approachesmentioning

confidence: 99%

See 3 more Smart Citations

Composing Noun Phrase Vector Representations

Kalouli¹,

Paiva²,

Crouch³

2019

Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)

View full text Add to dashboard Cite

Vector representations of words have seen an increasing success over the past years in a variety of NLP tasks. While there seems to be a consensus about the usefulness of word embeddings and how to learn them, it is still unclear which representations can capture the meaning of phrases or even whole sentences. Recent work has shown that simple operations outperform more complex deep architectures. In this work, we propose two novel constraints for computing noun phrase vector representations. First, we propose that the semantic and not the syntactic contribution of each component of a noun phrase should be considered, so that the resulting composed vectors express more of the phrase meaning. Second, the composition process of the two phrase vectors should apply suitable dimensions' selection in a way that specific semantic features captured by the phrase's meaning become more salient. Our proposed methods are compared to 11 other approaches, including popular baselines and a neural net architecture, and are evaluated across 6 tasks and 2 datasets. Our results show that these constraints lead to more expressive phrase representations and can be applied to other state-of-the-art methods to improve their performance.

show abstract

Section: Relevant Workmentioning

confidence: 99%

Section: Evaluation Of the Constraints 41 Compared Approachesmentioning

confidence: 99%

Section: Evaluation Of the Constraints 41 Compared Approachesmentioning

confidence: 99%

Section: Evaluation Of the Constraints 41 Compared Approachesmentioning

confidence: 99%

See 2 more Smart Citations

Composing Noun Phrase Vector Representations

Kalouli¹,

Paiva²,

Crouch³

2019

Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)

View full text Add to dashboard Cite

show abstract

“…Similarity evaluation needs to focus on how well a model is able to recover human similarity intuitions expressed as groupings, possibly around their focal points. We propose to treat it as a soft multi-class clustering problem (White et al, 2015), where two entities belong to the same class if there is a similarity judgement for them (e.g. apple and banana are similar because they are fruits) and the strength is proportional to the number of such judgements, so we could express that apple is more a fruit than it is a company.…”

Section: Dataset Constructionmentioning

confidence: 99%

A Proposal for Linguistic Similarity Datasets Based on Commonality Lists

Milajevs

Griffiths

2016

Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP

View full text Add to dashboard Cite

Similarity is a core notion that is used in psychology and two branches of linguistics: theoretical and computational. The similarity datasets that come from the two fields differ in design: psychological datasets are focused around a certain topic such as fruit names, while linguistic datasets contain words from various categories. The later makes humans assign low similarity scores to the words that have nothing in common and to the words that have contrast in meaning, making similarity scores ambiguous. In this work we discuss the similarity collection procedure for a multi-category dataset that avoids score ambiguity and suggest changes to the evaluation procedure to reflect the insights of psychological literature for word, phrase and sentence similarity. We suggest to ask humans to provide a list of commonalities and differences instead of numerical similarity scores and employ the structure of human judgements beyond pairwise similarity for model evaluation. We believe that the proposed approach will give rise to datasets that test meaning representation models more thoroughly with respect to the human treatment of similarity.

show abstract

Making service continuity smarter with artificial intelligence: An approach and its evaluation

2023

View full text Add to dashboard Cite

Service continuity entails establishing an observable and explainable continuum between customer experience and service operations. Such continuum is currently established manually, via service customer management operations (such as service incident management (IM)) often resulting in time‐consuming, human‐detrimental, and error‐prone activities. Conversely, artificial intelligence (AI) is rapidly emerging as an automated enabler towards handling the discontinuities in the aforementioned critical business tasks. Consequently, the emerging topic of AI‐driven incident management (AIIM) addresses practices and tools to resolve incidents by means of AI‐enabled organizational processes and methodologies. Our conjecture is that AIIM could reduce unplanned interruptions of service and let customers resume their work as quick as possible. While several techniques were presented in the literature to automatically identify the problems described in incident tickets by customers, this article focuses on the qualitative analysis and feature extraction off of the provided descriptions. When an incident ticket does not describe properly the problem, the analyst must ask the customer for additional details which could require several long‐lasting interactions. This article proposes ACQUA , an AIIM approach to automatically assess the quality of ticket descriptions with the goals of removing the need of additional communications and guiding the customers to properly describe the incident. A preliminary evaluation of ACQUA was performed on a dataset provided by a large bank in Europe, showing promising results and a boost of 13% in ticket resolution times and connected service continuity.

show abstract

How Well Sentence Embeddings Capture Meaning

Cited by 45 publications

References 8 publications

Composing Noun Phrase Vector Representations

Composing Noun Phrase Vector Representations

A Proposal for Linguistic Similarity Datasets Based on Commonality Lists

Making service continuity smarter with artificial intelligence: An approach and its evaluation

Contact Info

Product

Resources

About