Efficient Compression Technique for Sparse Sets

Abstract

Recent technological advancements have led to the generation of huge amounts of data over the web, such as text, image, audio and video. Needless to say, most of this data is high dimensional and sparse, consider, for instance, the bag-of-words representation used for representing text. O en, an e cient search for similar data points needs to be performed in many applications like clustering, nearest neighbour search, ranking and indexing. Even though there have been signi cant increases in computational power, a simple brute-force similarity-search on such datasets is ine cient and at times impossible. us, it is desirable to get a compressed representation which preserves the similarity between data points. In this work, we consider the data points as sets and use Jaccard similarity as the similarity measure. Compression techniques are generally evaluated on the following parameters -1) Randomness required for compression, 2) Time required for compression, 3) Dimension of the data a er compression, and 4) Space required to store the compressed data. Ideally, the compressed representation of the data should be such, that the similarity between each pair of data points is preserved, while keeping the time and the randomness required for compression as low as possible.Recently, Pratap and Kulkarni [11], suggested a compression technique for compressing high dimensional, sparse, binary data while preserving the Inner product and Hamming distance between each pair of data points. In this work, we show that their compression technique also works well for Jaccard similarity. We present a theoretical proof of the same and complement it with rigorous experimentations on synthetic as well as real-world datasets. We also compare our results with the state-of-the-art "min-wise independent permutation", and show that our compression algorithm achieves almost equal accuracy while signi cantly reducing the compression time and the randomness. Moreover, a er compression our compressed representation is in binary form as opposed to integer in case of min-wise permutation, which leads to a signi cant reduction in search-time on the compressed data.

Dimensionality Reduction for Categorical Data

“…is that it is impractical for large n or large c followed by a technical annoyance that some OHE implementations do not preserve the Hamming distances for sparse vectors (see illustration in Figure 1). Hence, this encoding is used in conjunction with problem-specific feature selection or followed by dimensionality reduction from binary to binary vectors [17], [18], [19]. The latter is a viable heuristic that we wanted to improve upon by allowing non-binary compressed vectors (see Appendix A for a quick analysis of OHE followed by a state-of-the-art binary compression).…”

Bit-string representation of a fingerprint image by normalized local structures

“…According to [45], [46], bit-string sparsity is beneficial for compression without accuracy performance degradation. In this study, we employed the binary compression scheme (BCS) introduced in [45], [46] for bit-string compression. Fig.…”

Efficient Sketching Algorithm for Sparse Binary Data

“…Our proposed algorithm is very similar in nature to the BCS algorithm [22], [23], which suggests a randomized bucketing algorithm where each index of the input is randomly assigned to one of the O(ψ 2 ) buckets; ψ denotes the sparsity of the dataset. The sketch of an input vector is obtained by computing the parity of the bits fallen in each bucket.…”

“…For Cosine Similarity, we compare BinSketch with SimHash [10], CBE [27] -a faster variant of SimHash, MinHash [26], using DOPH [24] in the algorithm of [26] instead of MinHash. For the Inner Product, BCS [23], Asymmetric MinHash [26], and Asymmetric DOPH -using DOPH [24] in [26], were the competing algorithms. In all these similarity measures, for sparse binary datasets, our proposed algorithm is faster, while simultaneously offering almost a similar performance as compared to the baselines.…”