Pattern Sampling in Distributed Databases

Diop, Lamine; Diop, Cheikh Talibouya; Giacometti, Arnaud; Soulet, Arnaud

doi:10.1007/978-3-030-54832-2_7

Cited by 4 publications

(22 citation statements)

References 17 publications

(27 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the first proposition of pattern sampling method [3] in 2009, numerous algorithms are proposed for output pattern sampling [15,4,16,17,18,19,20,21,9,22,23,11,10,24]. The types of these methods can be grouped into four main classes: random work (MCMC), SAT framework, multi-steps and reservoir sampling.…”

Section: Pattern Sampling Techniquesmentioning

confidence: 99%

“…Its usefulness has been widely demonstrated in recent years in many areas like feature classification [4], outlier detection [5] or interactive discovery [6,7,8]. It has also been applied to several pattern languages such as graphs [3], itemsets [9,4,10] and sequences [11,12].…”

Section: Introductionmentioning

confidence: 99%

“…In the field of pattern mining, it is also very important to take into account the size of the database which can be an obstacle when the data must be loaded into memory. Indeed, with the output sampling methods of the state-of-the-art, it is a necessity that the entire database be stored in memory except using a decentralized approach [10] where the data are natively decentralized in different sites. In our case, we assume that all transactions are available in a single and unique machine where the user needs to run a sampling algorithm.…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, we present a new method of output pattern sampling that is based on a compressed data structure. Our main objective is therefore to propose a generic output pattern sampling algorithm which extracts patterns proportionally to a length-based utility measure [10] from a trie of occurrences.…”

Section: Introductionmentioning

confidence: 99%

“…Section 6 theoretically analyses our method by detailing the time of building a trie of occurrences and that of drawing a pattern by TPSampling. Section 7 presents the experimental results of our approach by comparing them with those of the twostep method algorithm [4] combining length-based utility measures [10]. Finally, Section 8 concludes this paper and gives some perspectives.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Trie-based Output Space Itemset Sampling

Diop

Giacometti

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Itemset mining methods are techniques to discover relevant patterns in transactional databases. The first methods, called constrained-based pattern mining, are based on exhaustive pattern mining techniques which consists in returning all itemsets that satisfy a given constraint. The main issues that hinder their efficiency are the pattern explosion and the difficulty for a user to set the threshold value. To solve this problem, methods which return the most interesting patterns, called top-k, are also proposed, but they tend to a lack of diversity, a challenging issue for interactive pattern mining. However, interactive pattern mining is based on fast methods that respond effectively to user demand. To overcome all these problems, output pattern sampling is proposed in order to draw quickly a set of interesting patterns while guaranteeing a good diversity. Pattern sampling techniques are probabilistic methods that aim to draw a set of interesting patterns where each pattern is drawn with a probability proportional to a given interestingness measure. Nowadays, there are several measures that a user can test when interacting with the same database. In that case, the system should last a few time to take into account the new utility measures while guaranteeing an exact draw. So, the cost in time of utility change can be a real problem of output pattern sampling techniques in large databases. In addition, with the current sampling methods, it is necessary to store all data in memory and this storage is prohibitive for large data. To solve these problems, this paper deals with how to structure the data for the purpose of output pattern sampling under length-based utility measure in large transactional databases. So, we revisit the trie structure initially proposed by D. Knuth to enrich it and then no longer have need (i) to access the data to sample because the patterns will be directly taken from the enriched trie, (ii) to reprocess the entire dataset when utility changes. The computation of the value of a length-based utility measure is based on the lengths of all the patterns that are present in the database. So, we define a new structure of trie called trie of occurrences, built by our first algorithm TPSpace (Trie-based Pattern Space), which materializes all the occurrences of the patterns in the database. The data compression comes from a factorization of the information via the prefixes of the patterns. The particularly remarkable result is that, by definition, the trie of occurrences is the same for any length-based utility measure provided that the same values are kept for the minimum and maximum length constraints. We then describe TPSampling (Trie-based Pattern Sampling) which performs the sampling by drawing patterns according to a length-based utility measure from the trie of occurrences. This paper is completed by the complexity analysis in memory and in time of the method and experiments on benchmark datasets. TPSampling is competitive with the two-step approach to sample following a given interestingness measure but, as expected, it is more particularly advantageous if several utility measures are used thanks to the generic preprocessing. TPSampling is $10^5$ times faster than Two-Step for reprocessing in utility change.

show abstract

Section: Pattern Sampling Techniquesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Trie-based Output Space Itemset Sampling

Diop

Giacometti

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Parallelization of Sequential Pattern Sampling

Diop

Ba²

2021

2021 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

Trie-based Output Space Itemset Sampling

Diop

Giacometti

et al. 2022

2022 IEEE International Conference on Big Data (Big Data)

Self Cite

View full text Add to dashboard Cite

Itemset mining methods are techniques to discover relevant patterns in transactional databases. The first approach, called constrained-based pattern mining, is based on exhaustive pattern mining techniques which consist in returning all itemsets that satisfy a given constraint. The main issues that hinder their efficiency are the pattern explosion and the difficulty for a user to set the threshold value. To solve this problem, methods that return the most interesting patterns, called top-k, are also proposed, but they tend to lack diversity, a challenging issue for interactive pattern mining. However, interactive pattern mining is based on fast methods that respond effectively to user demand. To overcome all these problems, output pattern sampling is proposed to draw quickly a set of interesting patterns while guaranteeing a good diversity. Pattern sampling techniques are probabilistic methods that aim to draw a set of interesting patterns where each pattern is drawn with a probability proportional to a given interestingness measure. Nowadays, there are several measures that a user can test when interacting with the same database. In that case, the system should last a few times to consider the new utility measures while guaranteeing an exact draw. So, the cost in time of utility change can be a real problem for output pattern sampling techniques in large databases. In addition, with the current sampling methods, it is necessary to store all data in memory and this storage is prohibitive for large data. To solve these problems, this paper deals with how to structure the data for output pattern sampling under length-based utility measures in large transactional databases. So, we revisit the trie structure initially proposed by D. Knuth to enrich it and then no longer have the need (i) to access the data to sample because the patterns will be directly taken from the enriched trie, (ii) to reprocess the entire dataset when utility changes. The computation of the value of a length-based utility measure is based on the lengths of all the patterns that are present in the database. So, we define a new structure of trie called trie of occurrences, built by our first algorithm TPSpace (Trie-based Pattern Space), which materializes all the occurrences of the patterns in the database. Diop et alThe data compression comes from a factorization of the information via the prefixes of the patterns. The particularly remarkable result is that, by definition, the trie of occurrences is the same for any length-based utility measure provided that the same values are kept for the minimum and maximum length constraints. We then describe TPSampling (Trie-based Pattern Sampling) which performs the sampling by drawing patterns according to a length-based utility measure from the trie of occurrences. This paper is completed by the complexity analysis in memory and in time of the method and experiments on benchmark datasets. TPSampling is competitive with the twostep approach to sample following a given interestingness measure but, as expec...

show abstract

Pattern Sampling in Distributed Databases

Cited by 4 publications

References 17 publications

Trie-based Output Space Itemset Sampling

Trie-based Output Space Itemset Sampling

Parallelization of Sequential Pattern Sampling

Trie-based Output Space Itemset Sampling

Contact Info

Product

Resources

About