MCFsyn: A Multi-Party Set Reconciliation Protocol With the Marked Cuckoo Filter

Luo, Lailong; Zhao, Yawei; Rottenstreich, Ori; Richard, T. B.; Xue-shan, Luo

doi:10.1109/tpds.2021.3074440

Cited by 12 publications

(8 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where l is the length of the fingerprint. Typical CF variants include Adaptive CF [33], Consistent CF [34], Dynamic CF [35], Simplified CF [36], Vertical CF [37], Marked CF [11] and Vacuum filter [38]. They are investigated to enhance CF in terms of false positive rate, flexibility, key set extension, theoretical guarantee, space efficiency, set reconciliation and query throughput.…”

Section: B Cuckoo Filter and Its Variantsmentioning

confidence: 99%

“…Marked Cuckoo filter. Marked Cuckoo filter [11] is designed to represent the sets in each reconciliation participant. The MCF attaches a mark field in each slot to indicate which set(s) the stored fingerprint belongs to.…”

Section: B Cuckoo Filter and Its Variantsmentioning

confidence: 99%

“…Summary Cache [10] uses association query to achieve distributed caching. Association query is also involved in many other applications, including set reconciliations [11], indexing in data centers [12], distributed file system [13], database indexing [13] and data duplication [14].…”

Section: Introductionmentioning

confidence: 99%

“…The existing data structures, however, fail to achieve the four rationales properly and simultaneously. As shown in Table I, many sketch data structures are proposed to support the above tasks in terms of membership query (Bloom filter (BF) [26], Cuckoo filter (CF) [27]), association query (Marked Cuckoo filter (MCF) [11]), and multiplicity query (Counting Bloom filter (CBF) [28], Adaptive Bloom filter (ABF) [29], Elastic Sketch (ES) [17]. In addition to suffering from the inability of deletion, poor space efficiency, or parametric performance (i.e., filter works with additional filters or apriori knowledge, or degrades with the range of auxiliary information expands), these efforts mainly work for one specific query, thus do not appear amenable for general set queries.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A Shifting Filter Framework for Dynamic Set Queries

Luo

Guo

et al. 2023

IEEE/ACM Trans. Networking

View full text Add to dashboard Cite

Set query is a fundamental problem in computer systems. Plenty of applications rely on the query results of membership, association, and multiplicity. A traditional method that addresses such a fundamental problem is derived from Bloom filter. However, such methods may fail to support element deletion, require additional filters or apriori knowledge, making them unamenable to a high-performance implementation for dynamic set representation and query. In this paper, we envision a novel sketch framework that is multi-functional, non-parametric, space efficient, and deletable. As far as we know, none of the existing designs can guarantee such features simultaneously. To this end, we present a general shifting framework to represent auxiliary information (such as multiplicity, association) with the offset. Thereafter, we specify such design philosophy for a hash table horizontally at the slot level, as well as vertically at the bucket level. Theoretical and experimental results jointly demonstrate that our design works exceptionally well with three types of set queries under small memory.

show abstract

Section: B Cuckoo Filter and Its Variantsmentioning

confidence: 99%

Section: B Cuckoo Filter and Its Variantsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Shifting Filter Framework for Dynamic Set Queries

Luo

Guo

et al. 2023

IEEE/ACM Trans. Networking

View full text Add to dashboard Cite

show abstract

“…In data storage, for example, P2P networks store object content on each node through the sketch [12] . In data sharing, set synchronization can realize file information sharing between different hosts through the sketch [13] . In data security, the sketch naturally realizes data encryption operations by only representing the summary information of data through hash calculation, which ensures data privacy and security.…”

mentioning

confidence: 99%

Jump Filter: A Dynamic Sketch for Big Data Governance

Fu¹,

Luo²,

Guo³

et al. 2023

International Journal of Software and Informatics

View full text Add to dashboard Cite

With the rapid development of information technology, the volume of data maintains exponential growth, and the value of data is hard to mine. This brings significant challenges to the efficient management and control of each link in the data life cycle, such as data collection, cleaning, storage, and sharing. Sketch uses a hash table/matrix/bit vector to track the core characteristics of data, such as frequency, cardinality, and membership. This mechanism makes the sketch itself metadata, which has been widely used in sharing, transmission, update, and other scenarios. The rapid flow characteristic of big data has spawned dynamic sketches. The existing dynamic sketches have the advantage of expanding or shrinking the capacity with the size of the data stream by dynamically maintaining a list of probabilistic data structures in a chain or tree structure. However, there are problems with the excessive space overhead and time overhead increasing with the increase in the dataset cardinality. This paper designs a dynamic sketch for big data governance on the basis of the advanced jump consistent hash. This method can simultaneously achieve the space overhead that grows linearly with the dataset cardinality and the constant time overhead of data processing and analysis, effectively supporting the demanding big data processing and analysis tasks for big data governance. The validity and efficiency of the proposed method are verified by the comparison with traditional methods on various synthetic and natural datasets.

show abstract