PAN@FIRE: Overview of the Cross-Language !ndian Text Re-Use Detection Competition

Barrón-Cedeño, Alberto; Rosso, Paolo; Devi, Sobha Lalitha; Clough, Paul; Stevenson, Mark

doi:10.1007/978-3-642-40087-2_6

Cited by 5 publications

(8 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Overall, the cross language text reuse detection results obtained on our corpus (best f 1 =0.552) are comparatively lower than for the METER corpus (best f 1 =0.664), which is a gold standard mono-lingual text reuse detection corpus (Clough, Gaizauskas, Piao, & Wilks, 2002). Moreover, the results are also low when compared with CL!TR corpus (best f 1 =0.600), which contains cross language text reuse cases for the English-Hindi language pair (Barrón-Cedeño, Rosso, Devi, Clough, & Stevenson, 2013). 13 The rationale being that our corpus contains real examples of paraphrased text whereas CL!TR is a simulated corpus, manually created by volunteers in a controlled environment, who were allowed to use online automatic tools to translate the text (from English to Hindi language) and then modify it.…”

Section: Resultsmentioning

confidence: 99%

“…The CL!TR 4 (Cross-Language Indian Text Reuse) corpus is the first of its kind developed specifically for the analysis of cross-language text reuse detection in the Hindi-English language pair at document level (Barrón-Cedeño, Rosso, Devi, Clough, & Stevenson, 2013). The suspicious documents it contains are in Hindi and the source documents in English language.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

CLEU ‐ A Cross‐language english‐urdu corpus and benchmark for text reuse experiments

Muneer¹,

Sharjeel

Iqbal

et al. 2018

Asso for Info Science & Tech

View full text Add to dashboard Cite

Text reuse is becoming a serious issue in many fields and research shows that it is much harder to detect when it occurs across languages. The recent rise in multi-lingual content on the Web has increased cross-language text reuse to an unprecedented scale. Although researchers have proposed methods to detect it, one major drawback is the unavailability of large-scale gold standard evaluation resources built on real cases. To overcome this problem, we propose a cross-language sentence/passage level text reuse corpus for the English-Urdu language pair. The Cross-Language English-Urdu Corpus (CLEU) has source text in English whereas the derived text is in Urdu. It contains in total 3,235 sentence/passage pairs manually tagged into three categories that is near copy, paraphrased copy, and independently written. Further, as a second contribution, we evaluate the Translation plus Mono-lingual Analysis method using three sets of experiments on the proposed dataset to highlight its usefulness. Evaluation results (f 1 =0.732 binary, f 1 =0.552 ternary classification) indicate that it is harder to detect cross-language real cases of text reuse, especially when the language pairs have unrelated scripts. The corpus is a useful benchmark resource for the future development and assessment of cross-language text reuse detection systems for the English-Urdu language pair.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

CLEU ‐ A Cross‐language english‐urdu corpus and benchmark for text reuse experiments

Muneer¹,

Sharjeel

Iqbal

et al. 2018

Asso for Info Science & Tech

View full text Add to dashboard Cite

show abstract

“…In the name of cross-fertilization across evaluation forums, 9 in 2011, we started to be involved in the organization of tracks at FIRE, most of them as PAN tracks at FIRE. Initially, we addressed the problem of text reuse (2011) and SN Computer Science similarity search (2012, 2013), both from a cross-language perspective [10,23,24]. In the former two tracks, datasets with texts in English, Gujarati, and Hindi were provided.…”

Section: Pan Lab Tracks At Firementioning

confidence: 99%

“…In 2019, in the framework of a track on author profiling and deception detection in Arabic, we organized a task on the identification of age, gender, and language variety from tweets. 10 In this chapter, we will present three author profiling shared tasks we have organized at FIRE, describing the resources that we created and made available to the research community, illustrating the obtained results and highlighting the main achievements.…”

Section: Pan Lab Tracks At Firementioning

confidence: 99%

Author Profiling Tracks at FIRE

Rosso

Rangel

2020

SN COMPUT. SCI.

Self Cite

View full text Add to dashboard Cite

Benchmarking activities are vital for fostering research and addressing new challenging problems. During the last 10 years of the FIRE initiative, we have been involved in the organization of more than ten tracks, with the aim of the creation of new resources in several languages that were made available to the research community. This allowed to compare the new several approaches on the same datasets. In this chapter, we will focus on the description of three author profiling tracks, on their data creation as well as the result analysis.

show abstract

“…Martinez also investigated the cases where Wikipedia is mainly used for copy and paste plagiarism cases [26]. Wikipedia articles are taken as source documents for generating cross-lingual plagiarism detection corpus for Hindi-English language pair [27].…”

Section: Collection Of Source Textsmentioning

confidence: 99%

Design and Development of a Large Cross-Lingual Plagiarism Corpus for Urdu-English Language Pair

Haneef

Nawab

Munir

et al. 2019

Scientific Programming

View full text Add to dashboard Cite

Cross-lingual plagiarism occurs when the source (or original) text(s) is in one language and the plagiarized text is in another language. In recent years, cross-lingual plagiarism detection has attracted the attention of the research community because a large amount of digital text is easily accessible in many languages through online digital repositories and machine translation systems are readily available, making it easier to perform cross-lingual plagiarism and harder to detect it. To develop and evaluate cross-lingual plagiarism detection systems, standard evaluation resources are needed. The majority of earlier studies have developed cross-lingual plagiarism corpora for English and other European language pairs. However, for Urdu-English language pair, the problem of cross-lingual plagiarism detection has not been thoroughly explored although a large amount of digital text is readily available in Urdu and it is spoken in many countries of the world (particularly in Pakistan, India, and Bangladesh). To fulfill this gap, this paper presents a large benchmark cross-lingual corpus for Urdu-English language pair. The proposed corpus contains 2,395 source-suspicious document pairs (540 are automatic translation, 539 are artificially paraphrased, 508 are manually paraphrased, and 808 are nonplagiarized). Furthermore, our proposed corpus contains three types of cross-lingual examples including artificial (automatic translation and artificially paraphrased), simulated (manually paraphrased), and real (nonplagiarized), which have not been previously reported in the development of cross-lingual corpora. Detailed analysis of our proposed corpus was carried out using n-gram overlap and longest common subsequence approaches. Using Word unigrams, mean similarity scores of 1.00, 0.68, 0.52, and 0.22 were obtained for automatic translation, artificially paraphrased, manually paraphrased, and nonplagiarized documents, respectively. These results show that documents in the proposed corpus are created using different obfuscation techniques, which makes the dataset more realistic and challenging. We believe that the corpus developed in this study will help to foster research in an underresourced language of Urdu and will be useful in the development, comparison, and evaluation of cross-lingual plagiarism detection systems for Urdu-English language pair. Our proposed corpus is free and publicly available for research purposes.

show abstract

PAN@FIRE: Overview of the Cross-Language !ndian Text Re-Use Detection Competition

Cited by 5 publications

References 9 publications

CLEU ‐ A Cross‐language english‐urdu corpus and benchmark for text reuse experiments

CLEU ‐ A Cross‐language english‐urdu corpus and benchmark for text reuse experiments

Author Profiling Tracks at FIRE

Design and Development of a Large Cross-Lingual Plagiarism Corpus for Urdu-English Language Pair

Contact Info

Product

Resources

About