Semantic Specialization of Distributional Word Vector Spaces using                     Monolingual and Cross-Lingual Constraints

Mrkšić, Nikola; Vulić, Ivan; Séaghdha, Diarmuid Ó; Leviant, Ira; Reichart, Roi; Gašić, Milica; Korhonen, Anna; Young, Steve

doi:10.1162/tacl_a_00063

Cited by 161 publications

(189 citation statements)

References 48 publications

(87 reference statements)

Supporting

Mentioning

185

Contrasting

Order By: Relevance

“…Further, a lot of syntactic and semantic parsing models recently successfully incorporated parameter sharing for training parsers in closely related languges (Duong et al, 2015;Ammar et al, 2016;Susanto and Lu, 2017;. In the domain of dialog managers, Mrkšić et al (2017) and Chen et al (2018) presented methods for crosslingual transfer for dialog state tracking.…”

Section: Related Workmentioning

confidence: 99%

Cross-lingual Transfer Learning for Multilingual Task Oriented Dialog

Schuster¹,

Gupta²,

Shah³

et al. 2019

Proceedings of the 2019 Conference of the North

200

231

View full text Add to dashboard Cite

One of the first steps in the utterance interpretation pipeline of many task-oriented conversational AI systems is to identify user intents and the corresponding slots. Since data collection for machine learning models for this task is time-consuming, it is desirable to make use of existing data in a high-resource language to train models in low-resource languages. However, development of such models has largely been hindered by the lack of multilingual training data. In this paper, we present a new data set of 57k annotated utterances in English (43k), Spanish (8.6k) and Thai (5k) across the domains weather, alarm, and reminder. We use this data set to evaluate three different cross-lingual transfer methods: (1) translating the training data, (2) using cross-lingual pre-trained embeddings, and (3) a novel method of using a multilingual machine translation encoder as contextual word representations. We find that given several hundred training examples in the the target language, the latter two methods outperform translating the training data. Further, in very low-resource settings, multilingual contextual word representations give better results than using cross-lingual static embeddings. We also compare the cross-lingual methods to using monolingual resources in the form of contextual ELMo representations and find that given just small amounts of target language data, this method outperforms all cross-lingual methods, which highlights the need for more sophisticated cross-lingual methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Cross-lingual Transfer Learning for Multilingual Task Oriented Dialog

Schuster¹,

Gupta²,

Shah³

et al. 2019

Proceedings of the 2019 Conference of the North

200

231

View full text Add to dashboard Cite

show abstract

“…Also note that unlike retro-fitting and similar techniques (Rothe and Schütze, 2015;Pilehvar and Collier, 2016;Mrkšić et al, 2017), our approach does not use any training corpus or pretrained input embeddings. The synset representations are trained on the WordNet graph alone.…”

Section: Related Workmentioning

confidence: 99%

Learning Graph Embeddings from WordNet-based Similarity Measures

Kutuzov¹,

Dorgham²,

Oliynyk³

et al. 2019

Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019)

View full text Add to dashboard Cite

We present path2vec, a new approach for learning graph embeddings that relies on structural measures of pairwise node similarities. The model learns representations for nodes in a dense space that approximate a given userdefined graph distance measure, such as e.g. the shortest path distance or distance measures that take information beyond the graph structure into account. Evaluation of the proposed model on semantic similarity and word sense disambiguation tasks, using various WordNetbased similarity measures, show that our approach yields competitive results, outperforming strong graph embedding baselines. The model is computationally efficient, being orders of magnitude faster than the direct computation of graph-based distances.

show abstract

“…WOZ2.0 consists of a total of 1200 dialogues, out of which 600 are for training, 200 for development and 400 for testing. [17] translated the WOZ.0 English data both to German and Italian using professional translators. We experiment on the three languages (English, German, Italian) of the WOZ2.0 dataset.…”

Section: Datasetsmentioning

confidence: 99%

Scalable Neural Dialogue State Tracking

Balaraman

Magnini

2019

2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

View full text Add to dashboard Cite

A Dialogue State Tracker (DST) is a key component in a dialogue system aiming at estimating the beliefs of possible user goals at each dialogue turn. Most of the current DST trackers make use of recurrent neural networks and are based on complex architectures that manage several aspects of a dialogue, including the user utterance, the system actions, and the slot-value pairs defined in a domain ontology. However, the complexity of such neural architectures incurs into a considerable latency in the dialogue state prediction, which limits the deployments of the models in real-world applications, particularly when task scalability (i.e. amount of slots) is a crucial factor. In this paper, we propose an innovative neural model for dialogue state tracking, named Global encoder and Slot-Attentive decoders (G-SAT), which can predict the dialogue state with a very low latency time, while maintaining high-level performance. We report experiments on three different languages (English, Italian, and German) of the WOZ2.0 dataset, and show that the proposed approach provides competitive advantages over state-of-art DST systems, both in terms of accuracy and in terms of time complexity for predictions, being over 15 times faster than the other systems.

show abstract

Semantic Specialization of Distributional Word Vector Spaces using Monolingual and Cross-Lingual Constraints

Cited by 161 publications

References 48 publications

Cross-lingual Transfer Learning for Multilingual Task Oriented Dialog

Cross-lingual Transfer Learning for Multilingual Task Oriented Dialog

Learning Graph Embeddings from WordNet-based Similarity Measures

Scalable Neural Dialogue State Tracking

Contact Info

Product

Resources

About