Schema Extraction for Deep Web Query Interfaces Using Heuristics Rules

Abstract

Along with the popularity of the world wide web, data volumes inside web databases have been increasing tremendously. These deep web contents, hidden behind the query interfaces, are of much better quality than those in the surface web. Internet users need to fill in query conditions in the HTML query interface and click the submit button to obtain deep web data. Many deep web contents related applications, like named entity attribute collection, topic-focused crawling, and heterogeneous data integration, are based on understanding schema of these query interfaces. The schema needs to cover mappings of input elements and labels, data types of valid input values, and range constraints of the input values. Additionally, to extract these hidden data, the schema needs to include many form submission related information, like cookies and action types. We design and implement a Heuristics-based deep web query interface Schema Extraction system (HSE). In HSE, texts surrounding elements are collected as candidate labels. We propose a string similarity function and use a dynamic similarity threshold to cleanse candidate labels. In HSE, elements, candidate labels, and new lines in the query interface are streamlined to produce its Interface Expression (IEXP). By combining the user's view and the designer's view, with the aid of semantic information, we build heuristic rules to extract schema from IEXP of query interfaces in the ICQ dataset. These rules are constructed through utilizing (1) the characteristics of labels and elements, and (2) the spatial, group, and range relationships of labels and elements. Supplemented with form submission related information, the extracted schemas are then stored in the XML format, so that they could be utilized in further applications, like schema matching and merging for federated query interface integration. The experimental results on the TEL-8 dataset illustrate that HSE produces effective performance.

An Efficient Mechanism for Deep Web Data Extraction Based on Tree-Structured Web Pattern Matching

“…Manually creating these ontologies, on the other hand, is a time-consuming and difficult undertaking. The author of [8] described an automatic extraction strategy for studying domain ontologies for semantic web from the deep web.…”

A Machine Learning Approach for Automated Filling of Categorical Fields in Data Entry Forms

“…The number of fields in these systems is comparable with or larger than the data entry forms used in the related work. For example, we calculated the average number of fields of data entry forms in the TEL-8 dataset, a manually collected dataset with 447 web forms (with no input instances), which is used in the literature on form filling [9,44]. In this dataset, each form has 6.39 fields on average.…”

Guest Editorial: Recent Trends in Reuse and Integration

“…Finally, there is work going on pertaining to the use of schemas for knowledge extraction (e.g., for web query interfaces) (Jou 2019). Schemas are randomizations, which may be instantiated to create constrained knowledge.…”