Cross-Lingual Learning-to-Rank with Shared Representations

Sasaki, Shota; Sun, Shuo; Schamoni, Shigehiko; Duh, Kevin; Inui, Kentaro

doi:10.18653/v1/n18-2073

Cited by 47 publications

(47 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cross-language transfer learning employing deep neural networks has widely been studied in the areas of natural language processing (Ma and Xia, 2014;Guo et al, 2015;Kim et al, 2017;Kann et al, 2017;Cotterell and Duh, 2017), speech recognition (Xu et al, 2014;Huang et al, 2013), and information retrieval (Vulić and Moens, 2015;Sasaki et al, 2018;Litschko et al, 2018). Learning the language structure (e.g., morphology, syntax) and transferring knowledge from the source language to the target language is the main underneath challenge, and has been thoroughly investigated for a wide variety of NLP applications, including sequence tagging (Yang et al, 2016;Buys and Botha, 2016), name entity recognition (Xie et al, 2018), dependency parsing (Tiedemann, 2015;Agić et al, 2014), entity coreference resolution and linking , sentiment classification (Zhou et al, 2015(Zhou et al, , 2016b, and question answering (Joty et al, 2017).…”

Section: Related Workmentioning

confidence: 99%

On Difficulties of Cross-Lingual Transfer with Order Differences: A Case Study on Dependency Parsing

Ahmad¹,

Zhang²,

Ma³

et al. 2019

Proceedings of the 2019 Conference of the North

View full text Add to dashboard Cite

Different languages might have different word orders. In this paper, we investigate crosslingual transfer and posit that an orderagnostic model will perform better when transferring to distant foreign languages. To test our hypothesis, we train dependency parsers on an English corpus and evaluate their transfer performance on 30 other languages. Specifically, we compare encoders and decoders based on Recurrent Neural Networks (RNNs) and modified self-attentive architectures. The former relies on sequential information while the latter is more flexible at modeling word order. Rigorous experiments and detailed analysis shows that RNN-based architectures transfer well to languages that are close to English, while self-attentive models have better overall cross-lingual transferability and perform especially well on distant languages. * Equal contribution. Listed by alphabetical order. † Corresponding author. Language Families Languages Afro-Asiatic Arabic (ar), Hebrew (he) Austronesian Indonesian (id) IE.Baltic Latvian (lv) IE.Germanic Danish (da), Dutch (nl), English (en), German (de), Norwegian (no), Swedish (sv) IE.Indic Hindi (hi) IE.Latin Latin (la) IE.Romance Catalan (ca), French (fr), Italian (it), Portuguese (pt), Romanian (ro), Spanish (es) IE.Slavic Bulgarian (bg), Croatian (hr), Czech (cs), Polish (pl), Russian (ru), Slovak (sk), Slovenian (sl), Ukrainian (uk) Japanese Japanese (ja) Korean Korean (ko) Sino-Tibetan Chinese (zh) Uralic Estonian (et), Finnish (fi)

show abstract

Section: Related Workmentioning

confidence: 99%

On Difficulties of Cross-Lingual Transfer with Order Differences: A Case Study on Dependency Parsing

Ahmad¹,

Zhang²,

Ma³

et al. 2019

Proceedings of the 2019 Conference of the North

View full text Add to dashboard Cite

show abstract

“…Furthermore, Vulić and Moens (2015) and Litschko et al (2018) use cross-lingual word embeddings to represent both queries and documents as vectors and perform IR by computing the cosine similarity. Schamoni et al (2014) and Sasaki et al (2018) also use an automatic process to build CLIR datasets from Wikipeida articles. Neural Learning to Rank.…”

Section: Related Workmentioning

confidence: 99%

Improving Low-Resource Cross-lingual Document Retrieval by Reranking with Deep Bilingual Representations

Zhang¹,

Westerfield

Shim

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

In this paper, we propose to boost lowresource cross-lingual document retrieval performance with deep bilingual query-document representations. We match queries and documents in both source and target languages with four components, each of which is implemented as a term interaction-based deep neural network with cross-lingual word embeddings as input. By including query likelihood scores as extra features, our model effectively learns to rerank the retrieved documents by using a small number of relevance labels for low-resource language pairs. Due to the shared cross-lingual word embedding space, the model can also be directly applied to another language pair without any training label. Experimental results on the MA-TERIAL dataset show that our model outperforms the competitive translation-based baselines on English-Swahili, English-Tagalog, and English-Somali cross-lingual information retrieval tasks.

show abstract

“…We then pass the extracted features to an interaction-based relevance matching layer, followed by softmax to obtain relevance probability output. This CNN feature extraction for CLIR is similar to (Sasaki et al, 2018).…”

Section: Baseline Approachesmentioning

confidence: 99%

“…In contrast, previous methods that directly model CLIR rely on large amounts of relevanceannotated data (Sasaki et al, 2018;Lavrenko et al, 2002;Bai et al, 2009;Sokolov et al, 2013). Other approaches use bilingual embeddings to represent text cross-lingually, but are not specifically optimized for CLIR (Vulic and Moens, 2015;Litschko et al, 2018).…”

Section: Introduction and Previous Workmentioning

confidence: 97%

Weakly Supervised Attentional Model for Low Resource Ad-hoc Cross-lingual Information Retrieval

Zhao¹,

Zbib²,

Jiang³

et al. 2019

Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019)

View full text Add to dashboard Cite

We propose a weakly supervised neural model for Ad-hoc Cross-lingual Information Retrieval (CLIR) from low-resource languages. Low resource languages often lack relevance annotations for CLIR, and when available the training data usually has limited coverage for possible queries. In this paper, we design a model which does not require relevance annotations, instead it is trained on samples extracted from translation corpora as weak supervision. This model relies on an attention mechanism to learn spans in the foreign sentence that are relevant to the query. We report experiments on two low resource languages: Swahili and Tagalog, trained on less than 100k parallel sentences each. The proposed model achieves 19 MAP points improvement compared to using CNNs for feature extraction, 12 points improvement from machine translationbased CLIR, and up to 6 points improvement compared to probabilistic CLIR models.

show abstract

Cross-Lingual Learning-to-Rank with Shared Representations

Cited by 47 publications

References 9 publications

On Difficulties of Cross-Lingual Transfer with Order Differences: A Case Study on Dependency Parsing

On Difficulties of Cross-Lingual Transfer with Order Differences: A Case Study on Dependency Parsing

Improving Low-Resource Cross-lingual Document Retrieval by Reranking with Deep Bilingual Representations

Weakly Supervised Attentional Model for Low Resource Ad-hoc Cross-lingual Information Retrieval

Contact Info

Product

Resources

About