Lash

Abstract

We propose LASH, a scalable, distributed algorithm for mining sequential patterns in the presence of hierarchies. LASH takes as input a collection of sequences, each composed of items from some application-specific vocabulary. In contrast to traditional approaches to sequence mining, the items in the vocabulary are arranged in a hierarchy: both input sequences and sequential patterns may consist of items from different levels of the hierarchy. Such hierarchies naturally occur in a number of applications including mining natural-language text, customer transactions, error logs, or event sequences. LASH is the first parallel algorithm for mining frequent sequences with hierarchies; it is designed to scale to very large datasets. At its heart, LASH partitions the data using a novel, hierarchy-aware variant of item-based partitioning and subsequently mines each partition independently and in parallel using a customized mining algorithm called pivot sequence miner. LASH is amenable to a MapReduce implementation; we propose effective and efficient algorithms for both the construction and the actual mining of partitions. Our experimental study on large real-world datasets suggest good scalability and run-time efficiency.

Data mining in distributed environment: a survey

“…After that, some distributed and parallel mining methods, such as MapReduce Distributed GSP (DGSP) and large‐scale frequent sequence mining (MG‐FSM), were proposed by extending the traditional SPM algorithms. With the consideration of motif, uncertain sequences, and hierarchies, the advanced parallel motif extractor (ACME); an iterative MapReduce framework, and LASH algorithm were also proposed for large‐scale distributed sequence mining. Other related algorithms for distributed sequential pattern mining are still developed in progress, such as the memory‐efficient distributed DP approach …”