An empirical study on using hidden markov model for search interface segmentation

Abstract

This paper describes a hidden Markov model (HMM) based approach to perform search interface segmentation. Automatic processing of an interface is a must to access the invisible contents of deep Web. This entails automatic segmentation, i.e., the task of grouping related components of an interface together. While it is easy for a human to discern the logical relationships among interface components, machine processing of an interface is difficult. In this paper, we propose an approach to segmentation that leverages the probabilistic nature of the interface design process. The design process involves choosing components based on the underlying database query requirements, and organizing them into suitable patterns. We simulate this process by creating an artificial designer in the form of a 2-layered HMM. The learned HMM acquires the implicit design knowledge required for segmentation. We empirically study the effectiveness of the approach across several representative domains of deep Web. In terms of segmentation accuracy, the HMM-based approach outperforms an existing state-of-the-art approach by at least 10% in most cases. Furthermore, our cross-domain investigation shows that a single HMM trained on data having varied and frequent design patterns can accurately segment interfaces from multiple domains.

Web-based closed-domain data extraction on online advertisements

“…Khare and An [7] rely on a Hidden Markov Model (HMM) to label major components of a web interface, such as text-labels, text-boxes, and selection lists, and extract information from the interface. The authors in [7] train different HMMs, one for each available template of a web page, using training data that are grouped according to the templates.…”

“…Khare and An [7] rely on a Hidden Markov Model (HMM) to label major components of a web interface, such as text-labels, text-boxes, and selection lists, and extract information from the interface. The authors in [7] train different HMMs, one for each available template of a web page, using training data that are grouped according to the templates. This approach is similar to ADEx, since ADEx constructs a decision tree classifier for extracting data from ads in a particular domain D using training ads data belonged to D. Even though HMM is effective for data extraction, it is slower than existing supervised algorithms, including the decision tree employed by ADEx.…”

Automatically Mapping and Integrating Multiple Data Entry Forms into a Database

“…We have developed an effective and efficient method for segmenting form elements into semantically related groups [15]. To address the form2db problem, we extend the previous segmentation technique to a tree extraction method.…”

“…The FormMapper system is built on our previous work [15,16] and extends it in many aspects: First, the FormMapper integrates multiple forms into a single database instead of creating individual database for each form. Second, we have attempted to implement the following requirements in the FormMapper: (i) the system can accept sophisticated forms as input, (ii) the system automatically captures the semantic relationships among form elements, (iii) the system automatically links form elements to the elements in the hidden database, (iv) the system automatically extends the hidden database for unmatched form elements, and (v) the system automatically generates mapping expressions between the form and the hidden database.…”

“…The tree extraction component leverages a machine learning technique, Hidden Markov Model (HMM) [25], for automatically extracting a tree structure from a data entry form. In a previous study, we have applied the HMM model to the problem of segmenting search interfaces [15]. To address the form2db problem, we extend the method to automatically extract a complete tree structure from a form.…”