Using HTML Tags to Improve Parallel Resources Extraction

Abstract

This paper proposes a new approach to extract parallel resources (including bilingual sentences and bilingual terms) from bilingual web pages, which have a primary language and a secondary language (the second language is often the translation to primary language). Our method is composed of four tasks: 1) parsing the web page into a DOM tree and segmenting inner texts of each node into series of monolingual snippets; 2) selecting adjacent snippet pairs in different languages and with higher translation scores as seeds for the next task; 3) constructing comprehensive wrappers from selected seeds, which save both HTML and surface formatting styles; 4) mining candidate instances and selecting good instances by their similarities with seeds. In this paper, we first propose to segment text by HTML tags, and select potential parallel resources by ranking all extracted candidates. According to the experimental results, our method can be applied to bilingual pages written in any other pair of languages. Experimental results also show that our approaches are effective in improving the parallel resources extraction. (Abstract)

Topical search engine for Internet of Things

“…A crawler is a program for which we specify a seed URL, and keep going based on that URL retrieving connected pages. 1 A page is parsed for additional URLs by URL Normalization, where these URLs are saved in storage for crawling [27], [28], and are used to retrieve the more available Web pages from a Web server. The process of crawling may be divided among multiple distributed crawlers.…”