XWRAP: an XML-enabled wrapper construction system for Web information sources

Liu, L.; Pu, Calton; Han, Weihong

doi:10.1109/icde.2000.839475

Cited by 266 publications

(149 citation statements)

References 7 publications

Supporting

Mentioning

142

Contrasting

Unclassified

Order By: Relevance

“…A lot of technologies and systems have been proposed about web data wrapper in recent year, such as WIEN [6], Muslea [4], TSIMMIS [3], XWRAP [1], ARIADEN [5] etc. Our contributions in the paper are (1) With the guidance of extracting schema, the generated wrapper can be more accurate and better reflect user requirements.…”

Section: Experimental Results and Conclusionmentioning

confidence: 99%

“…The main process of the wrapper generation consists of two steps: (1) obtaining initial mapping rules; (2) adopting induction learning algorithm in order to get extracting rules. Foe example, the mapping rule instance is Web_robots.Robot.Name=Html [0].Body [1]. When extracting data from HTML document into XML, the element of Name in the XML document can be generated by the rule, formatted as following <Name> Acme.Spider </Name> The induction learning algorithm is based on obtaining initial mapping rules and extracting schema.…”

Section: Rule Extraction Algorithmmentioning

confidence: 99%

“…However, HTML is a mark-up language originally designed for platform-independent formatting and displaying. Such Web pages are more suitable for human browsing, rather than consumption by computer programs [1]. How to extract the data on the Web into an integrated system is urgent work for many enterprise applications.…”

Section: Introductionmentioning

confidence: 99%

“…With the development of the Internet, the Web has become invaluable information source [1,2]. Enterprises publish information and goods price on the Web.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Automatic Generation of Wrapper for Data Extraction from the Web

Zhang

2003

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract.With the development of the Internet, the Web has become invaluable information source. In order to use this information for more than human browsing, web pages in HTML must be converted into a format meaningful to software programs. Wrappers have been a useful technique to convert HTML documents into semantically meaningful XML files. In this paper, we propose a data extraction approach based on extracting schema, which generates automatically a wrapper to extract data from an HTML document, and produces an XML document conforming to given DTD. After the user defines extraction data schema in the form of DTD, the wrapper is generated automatically with the induction and leaning algorithm. The experiment indicates that the approach can correctly extract the required data from the source document with high accuracy.

show abstract

Section: Experimental Results and Conclusionmentioning

confidence: 99%

Section: Rule Extraction Algorithmmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…With the development of the Internet, the Web has become invaluable information source [1,2]. Enterprises publish information and goods price on the Web.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Automatic Generation of Wrapper for Data Extraction from the Web

Zhang

2003

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Thus, adding a filter to a pattern extends the set of extracted targets, whereas imposing a condition on a filter restricts the set of targets. The same concept is used by XWRAP [11] in order to describe the so-called "declarative information extraction rules". These rules are described in XPath-like expressions and point to regions of the HTML document that contain data records.…”

Section: Step 2: Generation Of Filtersmentioning

confidence: 99%

Self-supervised Automated Wrapper Generation for Weblog Data Extraction

et al. 2013

View full text Add to dashboard Cite

Abstract. Data extraction from the web is notoriously hard. Of the types of resources available on the web, weblogs are becoming increasingly important due to the continued growth of the blogosphere, but remain poorly explored. Past approaches to data extraction from weblogs have often involved manual intervention and suffer from low scalability. This paper proposes a fully automated information extraction methodology based on the use of web feeds and processing of HTML. The approach includes a model for generating a wrapper that exploits web feeds for deriving a set of extraction rules automatically. Instead of performing a pairwise comparison between posts, the model matches the values of the web feeds against their corresponding HTML elements retrieved from multiple weblog posts. It adopts a probabilistic approach for deriving a set of rules and automating the process of wrapper generation. An evaluation of the model is conducted on a dataset of 2,393 posts and the results (92% accuracy) show that the proposed technique enables robust extraction of weblog properties and can be applied across the blogosphere for applications such as improved information retrieval and more robust web preservation initiatives.

show abstract

An effective and efficient Web content extractor for optimizing the crawling process

Uzun

Güner

Kılıçaslan

et al. 2013

Softw. Pract. Exper.

View full text Add to dashboard Cite

SUMMARY Classical Web crawlers make use of only hyperlink information in the crawling process. However, focused crawlers are intended to download only Web pages that are relevant to a given topic by utilizing word information before downloading the Web page. But, Web pages contain additional information that can be useful for the crawling process. We have developed a crawler, iCrawler (intelligent crawler), the backbone of which is a Web content extractor that automatically pulls content out of seven different blocks: menus, links, main texts, headlines, summaries, additional necessaries, and unnecessary texts from Web pages. The extraction process consists of two steps, which invoke each other to obtain information from the blocks. The first step learns which HTML tags refer to which blocks using the decision tree learning algorithm. Being guided by numerous sources of information, the crawler becomes considerably effective. It achieved a relatively high accuracy of 96.37% in our experiments of block extraction. In the second step, the crawler extracts content from the blocks using string matching functions. These functions along with the mapping between tags and blocks learned in the first step provide iCrawler with considerable time and storage efficiency. More specifically, iCrawler performs 14 times faster in the second step than in the first step. Furthermore, iCrawler significantly decreases storage costs by 57.10% when compared with the texts obtained through classical HTML stripping. Copyright © 2013 John Wiley & Sons, Ltd.

show abstract

XWRAP: an XML-enabled wrapper construction system for Web information sources

Cited by 266 publications

References 7 publications

Automatic Generation of Wrapper for Data Extraction from the Web

Automatic Generation of Wrapper for Data Extraction from the Web

Self-supervised Automated Wrapper Generation for Weblog Data Extraction

An effective and efficient Web content extractor for optimizing the crawling process

Contact Info

Product

Resources

About