Concept hierarchy based text database categorization in a metasearch engine environment

Abstract

Document categorization as a technique to improve the retrieval of useful documents has been extensively investigated. One important issue in a large-scale metasearch engine is to select text databases that are likely to contain useful documents f o r a given query. We believe that database categorization can be a potentially effective technique f o r good database selection, especially in the Internet environment where short queries are usually submitted. In this paper, we propose and evaluate several database categorization algorithms. This study indicates that while some document categorization algorithms could be adopted f o r database categorization, algorithms that take into consideration the special characteristics of databases may be more effective. Preliminary experimental results are provided to compare the proposed database categorization algorithms.

Categorizing web search results into meaningful and stable categories using fast-feature techniques

“…QProber classified Web-accessible databases by issuing a set of queries (query probes) to each database and analyzing the counts of the number of results to classify each database into a set of thematic categories [11]. Wang, Meng and Yu [27] used a similar approach, starting with the top 2 levels of the Yahoo! hierarchy.…”

Discovering Interesting Relationships among Deep Web Databases: A Source-Biased Approach

“…One of the first deep web crawlers for discovering and interacting with deep web databases was proposed in [32], where the complexity of interacting with web search interfaces was noted. More recently, there have been efforts to extract data from deep web databases [7] and to match deep web query interfaces [39,40,43], among others.…”

Information source selection for resource constrained environments

“…An alternative approach is taken by [2,4,11,21]. These algorithms obtain information by issuing queries to the information source and receiving document counts and/or document samples.…”

“…A similar approach to [11] is taken by Wang et al [21] that does not rely on accurate document counts. However, it still requires a manually constructed topic tree, as well as a description for each topic, rather than document samples for each topic.…”