Lossless Compression of Chemical Fingerprints Using Integer Entropy Codes Improves Storage and Retrieval

Abstract

Many modern chemoinformatics systems for small molecules rely on large fingerprint vector representations, where the components of the vector record the presence or number of occurrences in the molecular graphs of particular combinatorial features, such as labeled paths or labeled trees. These large fingerprint vectors are often compressed to much shorter fingerprint vectors using a lossy compression scheme based on a simple modulo procedure. Here we combine statistical models of fingerprints with integer entropy codes, such as Golomb and Elias codes, to encode the indices or the run-lengths of the fingerprints. After reordering the fingerprint components by decreasing frequency order, the indices are monotone increasing and the run-lenghts are quasi-monotone increasing, and both exhibit power-law distribution trends. We take advantage of these statistical properties to derive new efficient, lossless, compression algorithms for monotone integer sequences: Monotone Value (MOV) Coding and Monotone Length (MOL) Coding. In contrast with lossy systems that use 1,024 or more bits of storage per molecule, we can achieve lossless compression of long chemical fingerprints based on circular substructures in slightly over 300 bits per molecule, close to the Shannon entropy limit, using a MOL Elias Gamma code for run-lengths. The improvement in storage comes at a modest computational cost. Furthermore, because the compression is lossless, uncompressed similarity (e.g. Tanimoto) between molecules can be computed exactly from their compressed representations, leading to significant improvements in retrival performance, as shown on six benchmark datasets of drug-like molecules.

A Survey on Data Compression Methods for Biological Sequences

“…The following methods support the compression of FASTQ files. The GenCompress method [101], first, maps short sequences to a reference genome; then, it encodes the addresses of the short sequences, their length and their probable substitutions, using entropy coding algorithms, e.g., Golomb [97], Elias Gamma [102], MOV (Monotone Value) [103] or Huffman coding [59]. Similar to GenCompress, the G-SQZ scheme [104] employs Huffman coding; however, it does compression without altering the relative order.…”

Improving molecular fingerprint similarity via enhanced folding

“…The researching methods are to improve storage and retrieval times of fingerprints [27]. Instead of simply storing the fingerprints as a string of binary bits, the researchers also calculate a new fingerprint representation based on Golomb and Golomb-Rice Codes.…”

The Mastermind Attack on Genomic Data

“…Indeed, recent compression schemes have shown that it is effective to view a genomic string with respect to a compression scheme that represents a string in terms of its differences with a reference string, R (e.g., see [4]). That is, we can start from a reference string, R, which contains the most common components of a typical genomic string.…”

“…All the sequences were aligned to the reference sequence and, for each sequence, the indices of the location of each variation were recorded together with the type (substitution, insertion, deletion) and content of each variation. This step is also essential if one is interested in compressing the data [4], for example. Statistics for the number of substitutions, deletions, and insertions for this data set of 1000 mtDNA sequences is given in Table 1 Of the 1000 sequences, 453 have only substitution events with respect to the reference string, R = rCRS.…”