Conceptual analysis of parallel corpus collected from the Web

Abstract

As illustrated by the World Wide Web, the volume of information in languages other than English has grown significantly in recent years. This highlights the importance of multilingual corpora. Much effort has been devoted to the compilation of multilingual corpora for the purpose of cross-lingual information retrieval and machine translation. Existing parallel corpora mostly involve European languages, such as English-French and English-Spanish. There is still a lack of parallel corpora between European languages and Asian languages. In the authors' previous work, an alignment method to identify one-to-one Chinese and English title pairs was developed to construct an English-Chinese parallel corpus that works automatically from the World Wide Web, and a 100% precision and 87% recall were obtained. Careful analysis of these results has helped the authors to understand how the alignment method can be improved. A conceptual analysis was conducted, which includes the analysis of conceptual equivalent and conceptual information alternation in the aligned and nonaligned English-Chinese title pairs that are obtained by the alignment method. The result of the analysis not only reflects the characteristics of parallel corpora, but also gives insight into the strengths and weaknesses of the alignment method. In particular, conceptual alternation, such as omission and addition, is found to have a significant impact on the performance of the alignment method.

Multilingual chief complaint classification for syndromic surveillance: An experiment with Chinese chief complaints

“…However, as ICD-9 codes are primarily used for billing purposes, they are not always informative for syndromic surveillance [18,19]. As such, freetext CCs remain one of the most important data sources for syndromic surveillance [20].…”

“…The corpus-based approach [20,[25][26][27] analyzes large document collections (parallel or comparable corpora) to construct a statistical translation model. It has the potential to translate emerging terminologies.…”

Automatic extraction of translations from web-based bilingual materials

“…However, the use of the web as a bilingual or multilingual corpus for language studies, for natural language processing and for cross-language information retrieval (Chen & Nie, 2000;Resnik, 2003;Sigurbjörnsson, Kamps & Rijke, 2005;Chesñevar, Sabaté & Maguitman, 2006;Li & Yang, 2006;Wang et al, 2006) is only in its infant stage. The relatively recent exploration of the web as a bilingual or multi-lingual corpus was made possible by the rapid growth in the number of web pages, and the availability of vast quantities of web-based translation texts involving many language pairs.…”