Note: On the set-union knapsack problem

Nehme-Haily, David Antoine

doi:10.1002/1520-6750(199410)41:6<833::aid-nav3220410611>3.0.co;2-q

Cited by 88 publications

(71 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We refer to an MC-tree in a unit as segment to differentiate it from the concept of a complete MC-tree in the topology. The situation of multiple input streams and multiple output streams occurs on the task who has an input stream partitioned with Merge and an output stream partitioned with Split, a unit boundary will be set between this operator and its upstream Algorithm 3: PLANSTRUCTUREDTOPOLOGY(P, R, T ) Input: An initial plan P; The amount of available resources R; Topology T ; Output: Replication plan P; 1 usage = 0; Su ← Set of the units split from topology T ; 2 foreach Unit Ui ∈ Su do 3 Build segment set Gi; 4 while usage ≤ R do 5 Candidates ← ∅ ; 6 foreach Unit Ui ∈ Su do 7 foreach non-replicated segment gi ∈ Ui do 8 CGi ← {gi}; 9 if OFP = OFP∪CG i then 10 Conduct a BFS from Ui to traverse all the units:…”

Section: ) Algorithm For Structured Topologymentioning

confidence: 99%

Tolerating correlated failures in Massively Parallel Stream Processing Engines

Zhou

2016

2016 IEEE 32nd International Conference on Data Engineering (ICDE)

View full text Add to dashboard Cite

Abstract-Fault-tolerance techniques for stream processing engines can be categorized into passive and active approaches. A typical passive approach periodically checkpoints a processing task's runtime states and can recover a failed task by restoring its runtime state using its latest checkpoint. On the other hand, an active approach usually employs backup nodes to run replicated tasks. Upon failure, the active replica can take over the processing of the failed task with minimal latency. However, both approaches have their own inadequacies in Massively Parallel Stream Processing Engines (MPSPE). The passive approach incurs a long recovery latency especially when a number of correlated nodes fail simultaneously, while the active approach requires extra replication resources. In this paper, we propose a new faulttolerance framework, which is Passive and Partially Active (PPA). In a PPA scheme, the passive approach is applied to all tasks while only a selected set of tasks will be actively replicated. The number of actively replicated tasks depends on the available resources. If tasks without active replicas fail, tentative outputs will be generated before the completion of the recovery process. We also propose effective and efficient algorithms to optimize a partially active replication plan to maximize the quality of tentative outputs. We implemented PPA on top of Storm, an open-source MPSPE and conducted extensive experiments using both real and synthetic datasets to verify the effectiveness of our approach.

show abstract

Section: ) Algorithm For Structured Topologymentioning

confidence: 99%

Tolerating correlated failures in Massively Parallel Stream Processing Engines

Zhou

2016

2016 IEEE 32nd International Conference on Data Engineering (ICDE)

View full text Add to dashboard Cite

show abstract

“…We define a range set associated to a collection of itemsets and give an upper bound to its (empirical) VCdimension and a procedure to compute this bound, showing an interesting connection with the SetUnion Knapsack Problem (SUKP) [7]. To the best of our knowledge, ours is the first work to apply these techniques to the field of TFIs, and in general the first application of the sample complexity bound based on empirical VC-dimension to the field of data mining.…”

Section: Our Contributionsmentioning

confidence: 99%

“…Given the very large number of transactions in typical dataset and the fact that the number of itemsets in a transaction is exponential in its length, this method would be computationally too expensive. An upper bound to d (and therefore to EVC(R(C), D)) can be computed by solving a Set-Union Knapsack Problem (SUKP) [7] associated to C.…”

Section: Computing the Vc-dimensionmentioning

confidence: 99%

“…Assume that the i 's are labelled in sorted decreasing order: Complexity and runtime considerations. Solving the SUKP optimally is NP-hard in the general case, although there are known restrictions for which it can be solved in polynomial time using dynamic programming [7]. Since we have unit weights and unit profits, our SUKP is equivalent to the densest ksubhypergraph problem, which can not be approximated within a factor of 2 [42].…”

Section: Definition 4 ([7]mentioning

confidence: 99%

See 1 more Smart Citation

Finding the True Frequent Itemsets

Riondato

Vandin

2014

Proceedings of the 2014 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

Frequent Itemsets (FIs) mining is a fundamental primitive in knowledge discovery. It requires to identify all itemsets appearing in at least a fraction θ of a transactional dataset D. Often though, the ultimate goal of mining D is not an analysis of the dataset per se, but the understanding of the underlying process that generated it. Specifically, in many applications D is a collection of samples obtained from an unknown probability distribution π on transactions, and by extracting the FIs in D one attempts to infer itemsets that are frequently (i.e., with probability at least θ) generated by π, which we call the True Frequent Itemsets (TFIs). Due to the inherently stochastic nature of the generative process, the set of FIs is only a rough approximation of the set of TFIs, as it often contains a huge number of false positives, i.e., spurious itemsets that are not among the TFIs. In this work we design and analyze an algorithm to identify a thresholdθ such that the collection of itemsets with frequency at leastθ in D contains only TFIs with probability at least 1 − δ, for some user-specified δ. Our method uses results from statistical learning theory involving the (empirical) VC-dimension of the problem at hand. This allows us to identify almost all the TFIs without including any false positive. We also experimentally compare our method with the direct mining of D at frequency θ and with techniques based on widely-used standard bounds (i.e., the Chernoff bounds) of the binomial distribution, and show that our algorithm outperforms these methods and achieves even better results than what is guaranteed by the theoretical analysis.

show abstract

“…In the NP-hardness proofs we extend techniques used in [9,14,11,3], that are well suited for -behavior of functions but do not suffice for -behavior. The completeness is usually proved by reduction from CLIQUE using elementary graph transformations, a technique to which we also resort.…”

Section: This Workmentioning

confidence: 99%

The complexity of detecting fixed-density clusters

Holzapfel

Kosub

Maaß

et al. 2006

Discrete Applied Mathematics

View full text Add to dashboard Cite

We study the complexity of finding a subgraph of a certain size and a certain density, where density is measured by the average degree. Let : N → Q + be any density function, i.e., is computable in polynomial time and satisfies (k) k − 1 for all k ∈ N. Then -CLUSTER is the problem of deciding, given an undirected graph G and a natural number k, whether there is a subgraph of G on k vertices that has average degree at least (k). For (k) = k − 1, this problem is the same as the well-known CLIQUE problem, and thus NP-complete. In contrast to this, the problem is known to be solvable in polynomial time for (k) = 2. We ask for the possible functions such that -CLUSTER remains NP-complete or becomes solvable in polynomial time. We show a rather sharp boundary: CLUSTER is NP-complete if = 2 + (1/k 1− ) for some > 0 and has a polynomial-time algorithm for = 2 + O(1/k). The algorithm also shows that for = 2 + O(1/k 1−o(1) ), -CLUSTER is solvable in subexponential time 2 n o(1) .

show abstract

Note: On the set-union knapsack problem

Cited by 88 publications

References 9 publications

Tolerating correlated failures in Massively Parallel Stream Processing Engines

Tolerating correlated failures in Massively Parallel Stream Processing Engines

Finding the True Frequent Itemsets

The complexity of detecting fixed-density clusters

Contact Info

Product

Resources

About