Learning Translational and Knowledge-based Similarities from Relevance Rankings for Cross-Language Retrieval

Schamoni, Shigehiko; Hieber, Felix; Sokolov, Artem; Riezler, Stefan

doi:10.3115/v1/p14-2080

Cited by 18 publications

(21 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where W ∈ R P ×Q encodes a feature matrix (Bai et al, 2010;Schamoni et al, 2014). The value of f (•, •) is the prediction of the classifier given a target vector p and a vector of related features q.…”

Section: Baseline Features Systemmentioning

confidence: 99%

QUality Estimation from ScraTCH (QUETCH): Deep Learning for Word-level Translation Quality Estimation

Kreutzer

Schamoni

Riezler

2015

Proceedings of the Tenth Workshop on Statistical Machine Translation

Self Cite

View full text Add to dashboard Cite

This paper describes the system submitted by the University of Heidelberg to the Shared Task on Word-level Quality Estimation at the 2015 Workshop on Statistical Machine Translation. The submitted system combines a continuous space deep neural network, that learns a bilingual feature representation from scratch, with a linear combination of the manually defined baseline features provided by the task organizers. A combination of these orthogonal information sources shows significant improvements over the combined systems, and produces very competitive F 1-scores for predicting word-level translation quality.

show abstract

Section: Baseline Features Systemmentioning

confidence: 99%

QUality Estimation from ScraTCH (QUETCH): Deep Learning for Word-level Translation Quality Estimation

Kreutzer

Schamoni

Riezler

2015

Proceedings of the Tenth Workshop on Statistical Machine Translation

Self Cite

View full text Add to dashboard Cite

show abstract

“…See Figure 1 for an illustration. This data construction process is similar to (Schamoni et al, 2014) who made an English-German CLIR dataset, but ours is at a larger scale. Specifically, we use Wikipedia dumps released on August 23, 2017.…”

Section: Large-scale Clir Datasetmentioning

confidence: 97%

“…The intuition is that the first sentence is usually a well-defined summary of its corresponding article and should be thematically related for articles linked to it from another language. Similar to (Schamoni et al, 2014), title words from the query sentences are removed, because they may be present across different language editions. This deletion prevents the task from becoming an easy keyword matching task.…”

Section: Large-scale Clir Datasetmentioning

confidence: 99%

Cross-Lingual Learning-to-Rank with Shared Representations

Sasaki¹,

Sun²,

Schamoni

et al. 2018

Proceedings of the 2018 Conference of the North American Chapter Of the Association for Computational Linguistics: Hu

Self Cite

View full text Add to dashboard Cite

Cross-lingual information retrieval (CLIR) is a document retrieval task where the documents are written in a language different from that of the user's query. This is a challenging problem for data-driven approaches due to the general lack of labeled training data. We introduce a large-scale dataset derived from Wikipedia to support CLIR research in 25 languages. Further, we present a simple yet effective neural learning-to-rank model that shares representations across languages and reduces the data requirement. This model can exploit training data in, for example, Japanese-English CLIR to improve the results of Swahili-English CLIR.

show abstract

“…We leave the investigation of complex queries to future work. We want to emphasize that our mining pipeline is compatible with all query types; for example, we can use the first sentences of documents as queries (Schamoni et al, 2014;Sasaki et al, 2018) if desired. 5 Note that documents in different languages do not share document IDs.…”

Section: Design Choicesmentioning

confidence: 99%

CLIRMatrix: A massively large collection of bilingual and multilingual datasets for Cross-Lingual Information Retrieval

Sun¹,

Duh²

2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

We present CLIRMatrix, a massively large collection of bilingual and multilingual datasets for Cross-Lingual Information Retrieval extracted automatically from Wikipedia. CLIR-Matrix comprises (1) BI-139, a bilingual dataset of queries in one language matched with relevant documents in another language for 139×138=19,182 language pairs, and(2) MULTI-8, a multilingual dataset of queries and documents jointly aligned in 8 different languages. In total, we mined 49 million unique queries and 34 billion (query, document, label) triplets, making it the largest and most comprehensive CLIR dataset to date. This collection is intended to support research in end-to-end neural information retrieval and is publicly available at https: //github.com/ssun32/CLIRMatrix. We provide baseline neural model results on BI-139, and evaluate MULTI-8 in both singlelanguage retrieval and mix-language retrieval settings.

show abstract

Learning Translational and Knowledge-based Similarities from Relevance Rankings for Cross-Language Retrieval

Cited by 18 publications

References 17 publications

QUality Estimation from ScraTCH (QUETCH): Deep Learning for Word-level Translation Quality Estimation

QUality Estimation from ScraTCH (QUETCH): Deep Learning for Word-level Translation Quality Estimation

Cross-Lingual Learning-to-Rank with Shared Representations

CLIRMatrix: A massively large collection of bilingual and multilingual datasets for Cross-Lingual Information Retrieval

Contact Info

Product

Resources

About