Scalable Frequent Sequence Mining with Flexible Subsequence Constraints

Abstract

We study scalable algorithms for frequent sequence mining under flexible subsequence constraints. Such constraints enable applications to specify concisely which patterns are of interest and which are not. We focus on the bulk synchronous parallel model with one round of communication; this model is suitable for platforms such as MapReduce or Spark. We derive a general framework for frequent sequence mining under this model and propose the D-SEQ and D-CAND algorithms within this framework. The algorithms differ in what data are communicated and how computation is split up among workers. To the best of our knowledge, D-SEQ and D-CAND are the first scalable algorithms for frequent sequence mining with flexible constraints. We conducted an experimental study on multiple real-world datasets that suggests that our algorithms scale nearly linearly, outperform common baselines, and offer acceptable generalization overhead over existing, less general mining algorithms.

A Unified Framework for Frequent Sequence Mining with Subsequence Constraints

“…Parallel and distributed mining algorithms that support flexible constraints are important future work. A first step was recently taken by Renz-Wieland et al [42], who proposed distributed mining algorithms based on DESQ for platforms such as MapReduce and Spark. Another recent vein of work [20] investigates static FST analysis problems that ask if a given task can be distributed or not.…”