Introduction to Handling Uncertainty in Artificial Intelligence

BinDash 2.0: New MinHash Scheme Allows Ultra-fast and Accurate Genome Search and Comparisons

“…We implemented so called original densification and optimal densification in the first version of BinDash (Zhao, 2019). However, more densification strategies have been proposed since the publication of original BinDash, e.g., faster densification (Mai, et al, 2020), re-randomized densification (Li, et al, 2019), bidirectional densification (Jia, et al, 2021) with even better run time behavior. Specifically, faster densification improved the worse-case densification computational complexity in optimal densification from O(n + s^2) to O(n + s*log(s)) with the same average-case O(n + s) (Figure 1b and c) while re-randomized densification further improves accuracy for optimal densification at the cost of additional computation since rerun MinHash within previously empty bins after optimal densification is computationally expensive when there are many empty bins, see detailed complexity analysis for re-randomization densification (Li, et al, 2019).…”

“…Specifically, faster densification improved the worse-case densification computational complexity in optimal densification from O(n + s^2) to O(n + s*log(s)) with the same average-case O(n + s) (Figure 1b and c) while re-randomized densification further improves accuracy for optimal densification at the cost of additional computation since rerun MinHash within previously empty bins after optimal densification is computationally expensive when there are many empty bins, see detailed complexity analysis for re-randomization densification (Li, et al, 2019). BinDash 2 implemented all flavors and variants of MinHash presented in Broder (1997), Li and König (2010), Li, et al (2012), Shrivastava and Li (2014), Shrivastava and Li (2014), Shrivastava (2017) (Figure 1a and b) and Mai, et al (2020) (Figure 1a and c) with SIMD (see below). The implementation detail is presented in Supplementary Material.…”