Efficient Implicit Unsupervised Text Hashing using Adversarial Autoencoder

Doan, Khoa D.; Reddy, Chandan K.

doi:10.1145/3366423.3380150

Cited by 8 publications

(3 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Specifically, bit independence means each bit is independent of the others in the output distribution for all hash codes [14]. It can better preserve the original locality structure of the data [8] but may render it unsuitable to calculate the similarity as prior knowledge distillation methods directly. To address this property, we first explore the distribution of bits within one relevance set.…”

Section: Bit Mask Mechanismmentioning

confidence: 99%

“…Desirable hash codes have unique properties in distribution. For example, some semantic hashing models [8,23,37,65] tend to generate "bit independence" hash codes, resulting in each bit being independent of the others. This property can better preserve the original locality structure of the data but may render it unsuitable to calculate the distance between two hash codes as done in prior knowledge distillation methods [49,56,66], which are generally designed for real-value representations.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Bit-mask Robust Contrastive Knowledge Distillation for Unsupervised Semantic Hashing

He,

Huang,

Liu

et al. 2024

Proceedings of the ACM Web Conference 2024

View full text Add to dashboard Cite

Unsupervised semantic hashing has emerged as an indispensable technique for fast image search, which aims to convert images into binary hash codes without relying on labels. Recent advancements in the field demonstrate that employing large-scale backbones (e.g., ViT) in unsupervised semantic hashing models can yield substantial improvements. However, the inference delay has become increasingly difficult to overlook. Knowledge distillation provides a means for practical model compression to alleviate this delay. Nevertheless, the prevailing knowledge distillation approaches are not explicitly designed for semantic hashing. They ignore the unique search paradigm of semantic hashing, the inherent necessities of the distillation process, and the property of hash codes. In this paper, we propose an innovative Bit-mask Robust Contrastive knowledge Distillation (BRCD) method, specifically devised for the distillation of semantic hashing models. To ensure the effectiveness of two kinds of search paradigms in the context of semantic hashing, BRCD first aligns the semantic spaces between the teacher and student models through a contrastive knowledge distillation objective. Additionally, * Corresponding Authors This work is licensed under a Creative Commons Attribution International 4.0 License.

show abstract

Section: Bit Mask Mechanismmentioning

confidence: 99%

mentioning

confidence: 99%

Bit-mask Robust Contrastive Knowledge Distillation for Unsupervised Semantic Hashing

He,

Huang,

Liu

et al. 2024

Proceedings of the ACM Web Conference 2024

View full text Add to dashboard Cite

show abstract

“…Deep neural networks have recently made a number of breakthroughs and have had great success with wellknown applications and tasks ranging from computer vision (Krizhevsky, Sutskever, and Hinton 2012;He et al 2016) and natural language processing (Devlin et al 2018;Doan and Reddy 2020) to other popular areas such as games (Berner et al 2019;Silver et al 2017), computational advertising (Zhao et al 2020b;Xu et al 2021), and structural biology (AlQuraishi 2019; Doan et al 2021). However, the training of neural networks usually requires large datasets, and at the same time, the whole training process is labor-intensive and resource-intensive.…”

Section: Introductionmentioning

confidence: 99%

Copyright-Certified Distillation Dataset: Distilling One Million Coins into One Bitcoin with Your Private Key

Liu

Cui

Gu³

2023

AAAI

View full text Add to dashboard Cite

The rapid development of neural network dataset distillation in recent years has provided new ideas in many areas such as continuous learning, neural network architecture search and privacy preservation. Dataset distillation is a very effective method to distill large training datasets into small data, thus ensuring that the test accuracy of models trained on their synthesized small datasets matches that of models trained on the full dataset. Thus, dataset distillation itself is commercially valuable, not only for reducing training costs, but also for compressing storage costs and significantly reducing the training costs of deep learning. However, copyright protection for dataset distillation has not been proposed yet, so we propose the first method to protect intellectual property by embedding watermarks in the dataset distillation process. Our approach not only popularizes the dataset distillation technique, but also authenticates the ownership of the distilled dataset by the models trained on that distilled dataset.

show abstract