Knowledge Discovery and Measures of Interest

Abstract

Library ofCongress Cataloging-in-Publication DataHilderman, Robert 1. Knowledge discovery and measures of interestlby Robert 1. Hilderman, Howard 1. Hamilton. p. cm. -(The Kluwer international series in engineering and computer science;SECS 638) Includes bibliographical references and index. ISBN 978-1-4419-4913-4 ISBN 978-1-4757-3283-2 (eBook)Data mining algorithms can be broadly classified into two general areas: summarization and anomaly detection [71]. Summarization algorithms find concise descriptions of input data. For example, classificatory algorithms partition input data into disjoint groups. The results of such classification might be represented as a high-level summary, a decision tree, or a set of characteristic rules, as with C4.5 [112], DBLearn [58], and KID3 [110]. Anomaly-detection algorithms identify unusual features of data, such as combinations that occur with greater or lesser frequency than might be expected. For example, association algorithms find, from transaction records, sets of items that appear with 4 East 11 $275.00 3 A summary generated from the cross-product domain for the compound attribute Shape-Size-Colour corresponds to a unique combination of nodes from the DGGs associated with the individual attributes, where one node is selected from the DGG associated with each attribute. For example, given the sales transaction database shown in Table 1.1 (assume the Shape, Size, and Colour attributes have been selected for generalization) and the associated DGGs shown in Figure 1.3, one of the many possible summaries that can be generated is shown in Table 1.5. The summary in Table 1.5 is obtained by generalizing the Shape attribute to the ANY node and the Size attribute to the Package node, while the Colour attribute remains ungeneralized.The complexity of the DGGs is a primary factor determining the number of summaries that can be generated, and depends only upon the number of KNOWLEDGE DISCOVERY AND MEASURES OF INTERESTsatisfying X -+ Y, and I X II Y I / N is the number of tuples expected if X and Y were independent (Le., not associated).When RI = 0, then X and Y are statistically independent and the rule is not interesting. When RI > 0 (RI < 0), then X is positively (negatively) correlated to Y. The significance of the correlation between X and Y can be determined using the chi-square test for a 2 x 2 contingency table. Those rules which do not exceed a predetermined minimum significance threshold are determined to be the most interesting.

Reduction of Number of Association Rules with Inter Itemset Distance in Transaction Databases

“…Comparative studies of different interestingness measures were done in [36] and [37]. The use of all confidence as a correlation measure for generating interesting association rules was undertaken in [38] and [39].…”

Extending Rule-Based Classifiers to Improve Recognition of Imbalanced Classes

“…Generality, support, confidence, logical sufficiency or necessity are examples of widely approved and used measures. Their systematic review is available, e.g., in [15]. Below we present two measures: support and confidence of a rule, as we refer to them in the further text.…”

Measuring interestingness of discovered skewed patterns in data cubes

“…Utilitybased measures have been used for objective-oriented association mining (for example, [26,33]), with user-specified objectives. In addition, numerous interestingness measures for summaries have been proposed in the literature including diversity (e.g., [12,33]), conciseness and generality [5], peculiarity [22][23][24][25], and surprisingness/unexpectedness [9]. Most of these methods with the exception of [9,10,20,30,32] have been applied for identifying patterns that have been mined from a given static dataset as opposed to providing guidance for navigating through multi-dimensional data cubes of graphs, which is the focus of our paper.…”