Discovering Interesting Association Rules in the Web Log Usage Data

Abstract

The immense volume of web usage data that exists on web servers contains potentially valuable information about the behavior of website visitors. This information can be exploited in various ways, such as enhancing the effectiveness of websites or developing directed web marketing campaigns. In this paper we will focus on applying association rules as a data mining technique to extract potentially useful knowledge from web usage data.We conducted a comprehensive analysis of web usage association rules found on a website of an educational institution. Our experiments confirm that, prior to pruning, the set of generated association rules contained too many non-interesting rules, which made it very difficult for a user to find and exploit useful information. Many of these rules are a simple consequence of the high correlation between web pages due to their interconnectedness through the website link structure.We proposed and applied a set of basic pruning schemes to reduce the rule set size and to remove a significant number of non-interesting rules. This pruning method decreased the size of our experimental rule set by more than three times, making it much simpler to browse for truly interesting rules. The percentage of truly interesting rules, which can initiate a webmaster to actions that can potentially enhance the website and improve its browsing experience, in our resulting experimental rule set was 41%.The analysis of association rules in our case study confirmed the hypothesis that discovering interesting and potentially useful association rules in web usage data does not have to be a timeconsuming task and can lead to actions that increase the website's effectiveness.

Web Usage Association Rule Mining System

“…The file contains 397,741 web requests and can be found at http://www.vtsns.edu.rs/maja/insite2011. The data set is about 30 times larger than that used in Dimitrijevic and Bosnjak (2010). Our association rule discovery software proved to be efficient and completed each test run within seconds.…”

“…In a previous research (Dimitrijevic & Bosnjak, 2010) the open source (Weka 3) data mining software was used for discovering association rules in web log data. However, Weka does not support web log mining in an efficient and natural way, while it is better suited for relational database mining.…”

“…In Dimitrijevic and Bosnjak (2010) pruning techniques based on the schemes for rule clustering and removing irrelevant rules were used. We now propose a rule pruning method based on avoiding rules that contain directly linked pages.…”

“…We used Weka in our previous research (Dimitrijevic & Bosnjak, 2010), but ran into several obstacles. We found that Weka did not contain an integrated set of tools to support all phases of web usage mining.…”

Survey on Apriori Algorithm for Market Analysis

“…To use association rule mining without support threshold [14][15][16] [17], another constraint such as similarity or confidence pruning is usually introduced. However, the coincidental item set problem had not been directly considered by any of these researches.…”