Closed sequential pattern mining for sitemap generation

Abstract

A sitemap represents an explicit specification of the design concept and knowledge organization of a website and is therefore considered as the website’s basic ontology. It not only presents the main usage flows for users, but also hierarchically organizes concepts of the website. Typically, sitemaps are defined by webmasters in the very early stages of the website design. However, during their life websites significantly change their structure, their content and their possible navigation paths. Even if this is not the case, webmasters can fail to either define sitemaps that reflect the actual website content or, vice versa, to define the actual organization of pages and links which do not reflect the intended organization of the content coded in the sitemaps. In this paper we propose an approach which automatically generates sitemaps. Contrary to other approaches proposed in the literature, which mainly generate sitemaps from the textual content of the pages, in this work sitemaps are generated by analyzing the Web graph of a website. This allows us to: i) automatically generate a sitemap on the basis of possible navigation paths, ii) compare the generated sitemaps with either the sitemap provided by the Web designer or with the intended sitemap of the website and, consequently, iii) plan possible website re-organization. The solution we propose is based on closed frequent sequence extraction and only concentrates on hyperlinks organized in “Web lists”, which are logical lists embedded in the pages. These “Web lists” are typically used for supporting users in Web site navigation and they include menus, navbars and content tables. Experiments performed on three real datasets show that the extracted sitemaps are much more similar to those defined by website curators than those obtained by competitor algorithms.

Utility-driven Mining of Contiguous Sequences

“…In addition, the volume of mining results can be quite large, making it difficult for users to analyze and utilize them. To address this problem, several constraints (such as contiguous [21], top-k [38], and closed [39] constraints) have been proposed to obtain a concise and more meaningful subset of the complete set of sequential patterns. Among them, the contiguous constraint is significant when the underlying ordering of items in the sequence data must be emphasized.…”

Efficient Methods for Clickstream Pattern Mining on Incremental Databases

“…Sequential pattern mining is an essential technique of data mining, and has been widely applied in various fields such as market analysis [1], stocks [2], weather prediction [3], medical treatment [4]- [6], mobile robots [7], and so on. Recently, many methods based on sequential pattern mining were developed, such as those used to analyze intrusion detection systems (IDS) [8], generate sitemaps [9], extract all recency-based sequential patterns (RF-Miner) [10], and carry out Vehicle Trajectory Prediction (VTP) [11].…”