Hierarchical Web-Page Clustering via In-Page and Cross-Page Link Structures

Abstract

Abstract. Despite of the wide diversity of web-pages, web-pages residing in a particular organization, in most cases, are organized with semantically hierarchic structures. For example, the website of a computer science department contains pages about its people, courses and research, among which pages of people are categorized into faculty, staff and students, and pages of research diversify into different areas. Uncovering such hierarchic structures could supply users a convenient way of comprehensive navigation and accelerate other web mining tasks. In this study, we extract a similarity matrix among pages via in-page and crosspage link structures, based on which a density-based clustering algorithm is developed, which hierarchically groups densely linked webpages into semantic clusters. Our experiments show that this method is efficient and effective, and sheds light on mining and exploring web structures.

MenuMiner

“…Although the results are mixed and no method achieves a high F measure, the authors show that their approach is able to detect entry pages that are not found by other methods. In the experiments in [3] and [24] a large number of pages are clustered, but no metrics are used for evaluating the clustering quality. Instead the resulting clusters themselves are listed in tables and figures.…”

“…For example, a list of links related to a certain resource or a group of external links do not have the described characteristics. S-menus are the skeleton of the information architecture of sites that are based on menus 3 . Because of their invariability the function of s-menus is not only to provide paths through the information space but also to communicate the organization of the content.…”

“…For instance, the clustering method described in [3] considers "parallel links" -links that are siblings in the DOM tree of a page. Such links are likely to belong to the same navigation element.…”

“…Though research has focused on extracting hierarchies from web sites (e.g. [1], [2], [3]), no solution has been found yet that could compete with human judgment. The same applies for the problem of mining web site boundaries or subsites (e.g.…”

Summarisation of weighted networks

“…Networks are widely used to model various types of interactions or relationships between entities in a myriad of applications, such as social interactions between persons (e.g. Kimura, Saito, Nakano, and Motoda (2010)), hyperlinks between web pages (Lin, Yu, Han, and Liu (2010)), or interactions between proteins (e.g. Mamitsuka (2012)).…”