An efficient approach for mining sequential patterns using multiple threads on very large databases

Huynh, Bao; Trinh, Cuong; Huynh, Huy M.; Van, Thien-Trang; Vo, Bay; Snåšel, Václav

doi:10.1016/j.engappai.2018.06.009

Cited by 27 publications

(14 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…PREFIXSPAN [23] was proposed by Pei et al in 2001. The MCM-SPADE algorithm [14] was proposed by Huynh et al in 2018 to find sequential patterns by applying multiple threads. When performing on very big databases, the MCM-SPADE algorithm [14] reduces the execution time and memory usage through a parallel approach by using a multi-core processor architecture and CMAP data structure that is used to store the necessary information on item.…”

Section: Related Workmentioning

confidence: 99%

“…In the domain of data mining from a sequence database, exploiting sequential patterns is an essential task that has been extensively examined [1,3,4,8,9,10,11,14,17,23,27,35]. AprioriAll [1] was the first algorithm designed to solve the sequential-pattern mining problem.…”

Section: Introductionmentioning

confidence: 99%

“…AprioriAll [1] was the first algorithm designed to solve the sequential-pattern mining problem. It was proposed by Agrawal et al in 1995 and is also the basis for later algorithms such as GSP [27], SPADE [35], SPAM [3], FREESPAN [12], PREFIXSPAN [23], PRISM [11], and MCM-SPADE [14]. These algorithms have shown good performance with respect to sequence databases that have short frequent sequences.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An Efficient Method for Mining Top-K Closed Sequential Patterns

et al. 2020

Self Cite

View full text Add to dashboard Cite

The problem of exploiting Closed Sequential Patterns (CSPs) is an essential task in data mining, with many different applications. It is used to resolve the situations of huge databases or low minimum support (minsup) thresholds in mining sequential patterns. However, it is challenging and needs a lot of time to customize the minsup values for generating appropriate numbers of CSPs desired by users. To conquer this issue, the TSP algorithm for mining top-k CSPs was previously proposed, with k being a given parameter. The algorithm would return the k CSPs which have the highest support values in a database. However, its execution time and memory usage were high. In this paper, an algorithm named TKCS (Top-K Closed Sequences) is proposed to mine the top-k CSPs efficiently. To improve the execution time and memory usage, it uses a vertical bitmap database to represent data. Besides, it adopts some useful strategies in the process of exploiting the top-k CSPs such as: always choosing the sequential patterns with the greatest support values for generating candidate patterns and storing top-k CSPs in an ascending order of the support values to increase the minsup value more quickly. The empirical results show that TKCS has better performance than TSP for discovering the top-k CSPs in terms of both runtime and memory usage. INDEX TERMS closed sequential pattern; data mining; sequential pattern; top-k sequential patterns.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Efficient Method for Mining Top-K Closed Sequential Patterns

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…KDD Cup is an annual competition that focuses on data mining and knowledge discovery material [13]. KDD Cup which was held in 1999 discussed intrusion detection on computer networks.…”

Section: Kdd Cup 1999 Data Setmentioning

confidence: 99%

Implementation of CM-SPADE Algorithm In Building Denial of Service Detection System Model Using Snort

Nugroho¹,

Megasari²,

Junaeti³

et al. 2020

Proceedings of the Proceedings of the 7th Mathematics, Science, and Computer Science Education International Seminar, MSCEIS 20

View full text Add to dashboard Cite

Along with the increasing use of computers, business activities and data storage also use it. The increasing importance of the computer raises challenges and risks, especially in security of the computer system and network. Various ways have been done to improve computer network security, one of which is using the Intrusion Detection System (IDS). To maximize the IDS detection function that is able to detect unknown attacks, a data mining approach is used to create an IDS that uses anomaly-based detection techniques. In this study, a sequential pattern mining technique, the CM-SPADE algorithm, is used to generate IDS rules that can detect DoS attacks. Modeling and rules are made by applying the CM-SPADE algorithm to the KDD Cup 1999 data set. The results reveal that the implementation of the CM-SPADE algorithm is able to produce IDS rules that can detect DoS attacks with an accuracy rate of 97.976%.

show abstract

“…It introduces a proper data structure for mining patterns, enabling the mining process to be completed with a one-time scan. Huynh et al [23] proposed a parallel approach named MCM-SPADE (Multiple threads Co-occurence Map-SPADE) using a multi-core processor system for an SPM with a very large database. The frequent pattern mining expresses item information of databases as a binary form, so it has a limitation that it cannot consider real data’s non-binary character.…”

Section: Introductionmentioning

confidence: 99%

An Efficient Incremental Mining Algorithm for Discovering Sequential Pattern in Wireless Sensor Network Environments

Lyu

2018

Sensors

View full text Add to dashboard Cite

Wireless sensor networks (WSNs) are an important type of network for sensing the environment and collecting information. It can be deployed in almost every type of environment in the real world, providing a reliable and low-cost solution for management. Huge amounts of data are produced from WSNs all the time, and it is significant to process and analyze data effectively to support intelligent decision and management. However, the new characteristics of sensor data, such as rapid growth and frequent updates, bring new challenges to the mining algorithms, especially given the time constraints for intelligent decision-making. In this work, an efficient incremental mining algorithm for discovering sequential pattern (novel incremental algorithm, NIA) is proposed, in order to enhance the efficiency of the whole mining process. First, a reasoned proof is given to demonstrate how to update the frequent sequences incrementally, and the mining space is greatly narrowed based on the proof. Second, an improvement is made on PrefixSpan, which is a classic sequential pattern mining algorithm with a high-complexity recursive process. The improved algorithm, named PrefixSpan+, utilizes a mapping structure to extend the prefixes to sequential patterns, making the mining step more efficient. Third, a fast support number-counting algorithm is presented to choose frequent sequences from the potential frequent sequences. A reticular tree is constructed to store all the potential frequent sequences according to subordinate relations between them, and then the support degree can be efficiently calculated without scanning the original database repeatedly. NIA is compared with various kinds of mining algorithms via intensive experiments on the real monitoring datasets, benchmarking datasets and synthetic datasets from aspects including time cost, sensitivity of factors, and space cost. The results show that NIA performs better than the existed methods.

show abstract

An efficient approach for mining sequential patterns using multiple threads on very large databases

Cited by 27 publications

References 26 publications

An Efficient Method for Mining Top-K Closed Sequential Patterns

An Efficient Method for Mining Top-K Closed Sequential Patterns

Implementation of CM-SPADE Algorithm In Building Denial of Service Detection System Model Using Snort

An Efficient Incremental Mining Algorithm for Discovering Sequential Pattern in Wireless Sensor Network Environments

Contact Info

Product

Resources

About