Improvement of HITS-based algorithms on web documents

Abstract

In this paper, we present t wo w ays to improve the precision of HITS-based algorithms on Web documents. First, by analyzing the limitations of current HITS-based algorithms, we propose a new weighted HITS-based method that assigns appropriate weights to in-links of root documents. Then, we combine content analysis with HITS-based algorithms and study the e ects of four representative r e l e v ance scoring methods, VSM, Okapi, TLS, and CDR, using a set of broad topic queries. Our experimental results show that our weighted HITS-based method performs signi cantly better than Bharat's improved HITS algorithm. When we combine our weighted HITS-based method or Bharat's HITS algorithm with any o f the four relevance scoring methods, the combined methods are only marginally better than our weighted HITS-based method. Between the four relevancescoring methods, there is no signi cant quality di erence when they are combined with a HITS-based algorithm.

Detection and optimized disposal of near-duplicate pages

“…As for the page ranking, PageRank [11] , HITS [12], or other classical ranking algorithms can be used to rank the pages by the degree of relevance with the retrieval keywords, ensuring that users can get the information they need as quickly as possible.…”

Undue influence

“…Li et al [22] found that HITS is vulnerable to the "small-in-large-out" situation. Gyongyi et al describe a new algorithm, TrustRank, to combat Web spam [17].…”

“…However with the appearance of link farms, in which sites are densely interconnected, HITS is no longer robust [5,21,22,8]. 1 For example, the top 10 authorities generated by HITS for query weather are shown in Table 1.…”

The story picturing engine

“…Methods based on mutual reinforcement principle have been widely reported in literature especially in the domains of journal evaluation and more recently on Web search [4,14,17,21,13].…”

“…Google's pagerank of a particular Web-page is a measure of its standing based on its link structure [4]. In [17], modification of HITS by assigning a weight to each link based on textual similarities between pages has been found to perform better than the original HITS.…”