Learning Sparse Binary Code for Maximum Inner Product Search

Ma, Changyi; Yu, Fangchen; Yu, Yueyao; Li, Wenye

doi:10.1145/3459637.3482132

Cited by 4 publications

(3 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The hashing methods (Shrivastava and Li 2014;Neyshabur and Srebro 2015;Andoni et al 2015;Chen et al 2019;Ma et al 2021) usually partition the space using a similaritypreserving hash function, find the relevant buckets for a given query and score only the items in these buckets.…”

Section: Mipsmentioning

confidence: 99%

Query-Aware Quantization for Maximum Inner Product Search

Zhang

Lian

ZHANG

et al. 2023

AAAI

View full text Add to dashboard Cite

Maximum Inner Product Search (MIPS) plays an essential role in many applications ranging from information retrieval, recommender systems to natural language processing. However, exhaustive MIPS is often expensive and impractical when there are a large number of candidate items. The state-of-the-art quantization method of approximated MIPS is product quantization with a score-aware loss, developed by assuming that queries are uniformly distributed in the unit sphere. However, in real-world datasets, the above assumption about queries does not necessarily hold. To this end, we propose a quantization method based on the distribution of queries combined with sampled softmax. Further, we introduce a general framework encompassing the proposed method and multiple quantization methods, and we develop an effective optimization for the proposed general framework. The proposed method is evaluated on three real-world datasets. The experimental results show that it outperforms the state-of-the-art baselines.

show abstract

Section: Mipsmentioning

confidence: 99%

Query-Aware Quantization for Maximum Inner Product Search

Zhang

Lian

ZHANG

et al. 2023

AAAI

View full text Add to dashboard Cite

show abstract

“…Faiss 4 is a library for efficient similarity search and clustering of dense vectors. Faiss employs the MIPS (Maximum Inner Product Search) to find the document vectors with the highest inner product with the query vector (Mussmann and Ermon, 2016).…”

Section: A Appendixmentioning

confidence: 99%

Towards a Unified Framework for Reference Retrieval and Related Work Generation

Shi,

Gao,

Zhang

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

The task of related work generation aims to generate a comprehensive survey of related research topics automatically, saving time and effort for authors. Existing methods simplify this task by using human-annotated references in a large-scale scientific corpus as information sources, which is time-and cost-intensive. To this end, we propose a Unified Reference Retrieval and Related Work Generation Model (UR 3 WG), which combines reference retrieval and related work generation processes in a unified framework based on the large language model (LLM). Specifically, UR 3 WG first leverages the world knowledge of LLM to extend the abstract and generate the query for the subsequent retrieval stage. Then a lexicon-enhanced dense retrieval is proposed to search relevant references, where an importance-aware representation of the lexicon is introduced. We also propose multi-granularity contrastive learning to optimize our retriever. Since this task is not simply summarizing the main points in references, it should analyze the complex relationships and present them logically. We propose an instruction-tuning method to guide LLM to generate related work. Extensive experiments on two wide-applied datasets demonstrate that our UR 3 WG outperforms the state-of-the-art baselines in both generation and retrieval metrics.

show abstract

“…The sparse inner product (SIP) [1], [2], as the basis for sparse linear algebra including matrix multiplication, matrix-vector inner product, and matrix inversion, has shown an irreplaceable role in various applications, especially in accelerating large-scale machine learning (ML) where sparsity is intertwined. Take the classification task with 20Newsgroups dataset [3] as an example.…”

Section: Introductionmentioning

confidence: 99%

New Secure Sparse Inner Product with Applications to Machine Learning

Xu¹,

Xu²,

Ning³

et al. 2022

Preprint

View full text Add to dashboard Cite

Sparse inner product (SIP) has the attractive property of overhead being dominated by the intersection of inputs between parties, independent of the actual input size. It has intriguing prospects, especially for boosting machine learning on large-scale data, which is tangled with sparse data. In this paper, we investigate privacy-preserving SIP problems that have rarely been explored before. Specifically, we propose two concrete constructs, one requiring offline linear communication which can be amortized across queries, while the other has sublinear overhead but relies on the more computationally expensive tool. Our approach exploits state-of-the-art cryptography tools including garbled Bloom filters (GBF) and Private Information Retrieval (PIR) as the cornerstone, but carefully fuses them to obtain non-trivial overhead reductions. We provide formal security analysis of the proposed constructs and implement them into representative machine learning algorithms including k-nearest neighbors, naive Bayes classification and logistic regression. Compared to the existing efforts, our method achieves 2-50× speedup in runtime and up to 10× reduction in communication.

show abstract

Learning Sparse Binary Code for Maximum Inner Product Search

Cited by 4 publications

References 21 publications

Query-Aware Quantization for Maximum Inner Product Search

Query-Aware Quantization for Maximum Inner Product Search

Towards a Unified Framework for Reference Retrieval and Related Work Generation

New Secure Sparse Inner Product with Applications to Machine Learning

Contact Info

Product

Resources

About