HistoSketch: Fast Similarity-Preserving Sketching of Streaming Histograms with Concept Drift

Abstract

Abstract-Histogram-based similarity has been widely adopted in many machine learning tasks. However, measuring histogram similarity is a challenging task for streaming data, where the elements of a histogram are observed in a streaming manner. First, the ever-growing cardinality of histogram elements makes any similarity computation inefficient. Second, the concept-drift issue in the data streams also impairs the accurate assessment of the similarity. In this paper, we propose to overcome the above challenges with HistoSketch, a fast similarity-preserving sketching method for streaming histograms with concept drift. Specifically, HistoSketch is designed to incrementally maintain a set of compact and fixed-size sketches of streaming histograms to approximate similarity between the histograms, with the special consideration of gradually forgetting the outdated histogram elements. We evaluate HistoSketch on multiple classification tasks using both synthetic and real-world datasets. The results show that our method is able to efficiently approximate similarity for streaming histograms and quickly adapt to concept drift. Compared to full streaming histograms gradually forgetting the outdated histogram elements, HistoSketch is able to dramatically reduce the classification time (with a 7500x speedup) with only a modest loss in accuracy (about 3.5%).

Streaming histogram sketching for rapid microbiome analytics

“…However, rather than computing and storing a full k-mer spectrum after reading the sequence data, which is resource intensive (in terms of memory or disk space), we use the recently proposed histosketch data structure to maintain a set of fixed size sketches to approximate the overall k-mer spectrum as it is received from a data stream (Yang et al , 2017) . The histosketch has two properties making it suitable for this application, i .…”

“…We view the k-mer spectrum as a histogram, where k-mers from a microbiome sample are hashed uniformly across N bins and the frequency value of a bin corresponds to observed k-mer frequency. In order to incorporate both the bin and frequency (a weighted set) into the histosketch, we employ Consistent Weighted Sampling (CWS) to generate hash values for each histogram element, which ensures that the computational complexity of hashing is independent of bin frequency (Ioffe, 2010;Yang et al , 2017) .…”

“…To initialise S from V, first three independent variables are sampled from the CWS distributions: r i , j ∼ Gamma(2, 1), c i , j ∼ Gamma(2, 1) and β i , j ∼ Uniform(0, 1) for i ∈ E and j = 1, ..., Z. We then use Algorithm 1 of Yang et al for histosketch creation (Yang et al , 2017) . The sketch, S, and the corresponding hash values, A, are both kept as the histosketch (A allows for incremental sketch updating).…”

“…In addition to this update method, we can also utilise the gradual forgetting weights of the original histosketch implementation to adjust for changes in the underlying distribution (concept drift) (Koychev, 2000;Yang et al , 2017) . Prior to the update, uniform scaling is applied to the estimate frequency counts.…”

“…We utilise consistent weighted sampling to incorporate k-mer frequency information into the histosketch, allowing the use of weighted and standard Jaccard similarity for histosketch comparisons and sample retrieval (Ioffe, 2010) . Our method combines the recently proposed histogram sketching algorithm of Yang et al with count-min sketching of k-mer spectra and our recent implementation of LSH Forest indexing for microbiome searching (Yang et al , 2017;Zhang et al , 2014;Rowe and Winn, 2018) . We show our method to accurately cluster microbiome samples by sample type and demonstrate the utility of these histosketches to create and search microbiome sequence databases.…”