Learning Semantic Textual Similarity from Conversations

Yang, Yinfei; Yuan, Steve; Cer, Daniel; Kong, Sheng-yi; Constant, Noah; Pilar, Petr; Ge, Heming; Sung, Yun-Hsuan; Strope, Brian; Kurzweil, Ray

doi:10.48550/arxiv.1804.07754

Cited by 21 publications

(16 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To measure semantic similarity of text, various encoding methods have been developed which transform the text into a vector space [20,8,38,6]. Using the cosine similarity in conjunction with such an embedding space can ensure that similar text will be closer together.…”

Section: Feature Spacesmentioning

confidence: 99%

“…Adversarial gain can be measured across different feature spaces (and thus different manifolds). However, another appropriate method may be to learn a specific embeddings (feature) space for the problem at hand similarly to Yang et al [38] since well-generalized embedding spaces are difficult to create [9]. By learning a feature space which ensures a well-defined distance-based correlation between inputs and outputs, the distance assumption can more accurately measure whether an adversarial attack falls in the gain range where a mistake is more likely.…”

Section: Feature Spacesmentioning

confidence: 99%

See 1 more Smart Citation

Adversarial Gain

Henderson,

Sinha,

et al. 2018

Preprint

View full text Add to dashboard Cite

Adversarial examples can be defined as inputs to a model which induce a mistake -where the model output is different than that of an oracle, perhaps in surprising or malicious ways. Original models of adversarial attacks are primarily studied in the context of classification and computer vision tasks. While several attacks have been proposed in natural language processing (NLP) settings, they often vary in defining the parameters of an attack and what a successful attack would look like. The goal of this work is to propose a unifying model of adversarial examples suitable for NLP tasks in both generative and classification settings. We define the notion of adversarial gain: based in control theory, it is a measure of the change in the output of a system relative to the perturbation of the input (caused by the so-called adversary) presented to the learner. This definition, as we show, can be used under different feature spaces and distance conditions to determine attack or defense effectiveness across different intuitive manifolds. This notion of adversarial gain not only provides a useful way for evaluating adversaries and defenses, but can act as a building block for future work in robustness under adversaries due to its rooted nature in stability and manifold theory.

show abstract

Section: Feature Spacesmentioning

confidence: 99%

Section: Feature Spacesmentioning

confidence: 99%

Adversarial Gain

Henderson,

Sinha,

et al. 2018

Preprint

View full text Add to dashboard Cite

show abstract

“…Chit-chat agents, by contrast, might focus on coarse statistical regularities of dialogue data without accurately modeling the underlying "meaning"; but the data often covers a much wider space of natural language. For example, Twitter or Reddit chit-chat tasks (Li et al, 2016a;Yang et al, 2018;Mazaré et al, 2018) cover a huge spectrum of language and diverse topics. Chit-chat and goal-oriented dialogue are not mutually exclusive: when humans engage in chit-chat, their aim is to exchange information, or to elicit specific responses from their partners.…”

Section: Introductionmentioning

confidence: 99%

I love your chain mail! Making knights smile in a fantasy game world: Open-domain goal-oriented dialogue agents

Prabhumoye¹,

Li²,

Urbanek³

et al. 2020

Preprint

View full text Add to dashboard Cite

Dialogue research tends to distinguish between chit-chat and goal-oriented tasks. While the former is arguably more naturalistic and has a wider use of language, the latter has clearer metrics and a straightforward learning signal. Humans effortlessly combine the two, for example engaging in chit-chat with the goal of exchanging information or eliciting a specific response. Here, we bridge the divide between these two domains in the setting of a rich multi-player text-based fantasy environment where agents and humans engage in both actions and dialogue. Specifically, we train a goal-oriented model with reinforcement learning against an imitation-learned "chit-chat" model with two approaches: the policy either learns to pick a topic or learns to pick an utterance given the top-K utterances from the chit-chat model. We show that both models outperform an inverse model baseline and can converse naturally with their dialogue partner in order to achieve goals.

show abstract

“…Deep learning based dialogue systems have shown promising performance in many applications such as smart reply [10], conversation semantic embedding [41], human-computer interaction [8] and others [21,45,48]. Deep neural nets extract rich representations with high-level semantic information that are useful for message retrieval [39,41] and response generation [48] in conversations.…”

Section: Introductionmentioning

confidence: 99%

“…The dual encoder model [10,41] is widely used among various dialogue models especially for retrieving response messages, due to its simple structure and competitive computational speed. The dual encoder model consists of two separate encoders, which extract features for the dialogue context (e.g.…”

Section: Introductionmentioning

confidence: 99%

Toward Interpretability of Dual-Encoder Models for Dialogue Response Suggestions

Li¹,

Li²,

Prakash³

et al. 2020

Preprint

View full text Add to dashboard Cite

This work shows how to improve and interpret the commonly used dual encoder model for response suggestion in dialogue. We present an attentive dual encoder model that includes an attention mechanism on top of the extracted word-level features from two encoders, one for context and one for label respectively. To improve the interpretability in the dual encoder models, we design a novel regularization loss to minimize the mutual information between unimportant words and desired labels, in addition to the original attention method, so that important words are emphasized while unimportant words are de-emphasized. This can help not only with model interpretability, but can also further improve model accuracy. We propose an approximation method that uses a neural network to calculate the mutual information. Furthermore, by adding a residual layer between raw word embeddings and the final encoded context feature, word-level interpretability is preserved at the final prediction of the model. We compare the proposed model with existing methods for the dialogue response task on two public datasets (Persona and Ubuntu). The experiments demonstrate the effectiveness of the proposed model in terms of better Recall@1 accuracy and visualized interpretability.

show abstract

Learning Semantic Textual Similarity from Conversations

Cited by 21 publications

References 12 publications

Adversarial Gain

Adversarial Gain

I love your chain mail! Making knights smile in a fantasy game world: Open-domain goal-oriented dialogue agents

Toward Interpretability of Dual-Encoder Models for Dialogue Response Suggestions

Contact Info

Product

Resources

About