Joint Image-Text Hashing for Fast Large-Scale Cross-Media Retrieval Using Self-Supervised Deep Learning

Wu, Gengshen; Han, Jungong; Lin, Zijia; Ding, Guiguang; Zhang, Baochang; Ni, Qiang

doi:10.1109/tie.2018.2873547

Cited by 74 publications

(31 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Experimental results demonstrate that the proposed RGB-T salient object detection method performs better than the stateof-the-art methods, especially for those challenging scenes with poor illumination, complex background or low contrast. One possible future work is to ap-ply our saliency detector to industrial applications, such as image classification [72], object tracking [73], [74] and instance-level object retrieval [75], [76].…”

Section: Discussionmentioning

confidence: 99%

RGB-T Salient Object Detection via Fusing Multi-Level CNN Features

Zhang

Huang

Yang

et al. 2020

IEEE Trans. on Image Process.

Self Cite

167

View full text Add to dashboard Cite

RGB-induced salient object detection has recently witnessed substantial progress, which is attributed to the superior feature learning capability of deep convolutional neural networks (CNNs). However, such detections suffer from challenging scenarios characterized by cluttered backgrounds, low-light conditions and variations in illumination. Instead of improving RGB based saliency detection, this paper takes advantage of the complementary benefits of RGB and thermal infrared images. Specifically, we propose a novel end-to-end network for multimodal salient object detection, which turns the challenge of RGB-T saliency detection to a CNN feature fusion problem. To this end, a backbone network (e.g., VGG-16) is first adopted to extract the coarse features from each RGB or thermal infrared image individually, and then several adjacent-depth feature combination (ADFC) modules are designed to extract multi-level refined features for each single-modal input image, considering that features captured at different depths differ in semantic information and visual details. Subsequently, a multi-branch group fusion (MGF) module is employed to capture the crossmodal features by fusing those features from ADFC modules for a RGB-T image pair at each level. Finally, a joint attention guided bi-directional message passing (JABMP) module undertakes the task of saliency prediction via integrating the multi-level fused features from MGF modules. Experimental results on several public RGB-T salient object detection datasets demonstrate the superiorities of our proposed algorithm over the state-of-the-art approaches, especially under challenging conditions, such as poor illumination, complex background and low contrast.

show abstract

Section: Discussionmentioning

confidence: 99%

RGB-T Salient Object Detection via Fusing Multi-Level CNN Features

Zhang

Huang

Yang

et al. 2020

IEEE Trans. on Image Process.

Self Cite

167

View full text Add to dashboard Cite

show abstract

“…In the future, we will integrate scene understanding and scene parsing in our work to improve the performance. Besides, we will apply our salient object detector to facilitate the representation ability of existing deep networks [73], [74] and real-world applications, including image retrieval [75], [76] and image classification [77].…”

Section: Discussionmentioning

confidence: 99%

Deep Salient Object Detection With Contextual Information Guidance

Liu

Han

Zhang

et al. 2020

IEEE Trans. on Image Process.

Self Cite

View full text Add to dashboard Cite

Integration of multi-level contextual information, such as feature maps and side outputs, is crucial for Convolutional Neural Networks (CNNs) based salient object detection. However, most existing methods either simply concatenate multilevel feature maps or calculate element-wise addition of multilevel side outputs, thus failing to take full advantages of them. In this work, we propose a new strategy for guiding multilevel contextual information integration, where feature maps and side outputs across layers are fully engaged. Specifically, shallower-level feature maps are guided by the deeper-level side outputs to learn more accurate properties of the salient object. In turn, the deeper-level side outputs can be propagated to highresolution versions with spatial details complemented by means of shallower-level feature maps. Moreover, a group convolution module is proposed with the aim to achieve high-discriminative feature maps, in which the backbone feature maps are divided into a number of groups and then the convolution is applied to the channels of backbone feature maps within each group. Eventually, the group convolution module is incorporated in the guidance module to further promote the guidance role. Experiments on three public benchmark datasets verify the effectiveness and superiority of the proposed method over the state-of-the-art methods.

show abstract

“…To support fast similarity search in high-dimensional databases, various schemes have been proposed in recent decades [24]. The typical examples include M-tree [25], the VA-file [26], Hybrid tree [27], the iDistance [28] and Hashing [17,[29][30][31][32][33]. In [25], the authors proposed the height-balanced M-tree to organize and search large datasets from a generic metric space, where object proximity is defined by a distance function satisfying the positivity, symmetry and triangle inequality postulates.…”

Section: Multidimensional Indexing Structurementioning

confidence: 99%

“…In recent years, advanced hashing has been playing more and more important role in support of fast and effective multimedia information retrieval [30][31][32]. Consequently, a steady progress in the related field has been observed.…”

Section: Multidimensional Indexing Structurementioning

confidence: 99%

Toward efficient indexing structure for scalable content-based music retrieval

Shen

Mei

et al. 2019

Multimedia Systems

View full text Add to dashboard Cite

With advancement of various information processing and storage techniques, the scale of digital music collections has been growing at very fast speed during recent decades. To support high-quality content-based retrieval over such a large volume of music data, how to develop indexing structure with good effectiveness, efficiency and scalability becomes an important research issue. However, existing techniques mainly focus on improving query efficiency. Very few approaches have been proposed to address issues related to scalability and accuracy. In this study, we address the problem via introducing a novel indexing technique called effective music indexing framework (EMIF) to facilitate scalable and accurate music retrieval. It is designed based on a "classification-and-indexing" principle and consists of two main functionality modules: (1) music classification-a novel semantic-sensitive classification to identify an input song's category and (2) indexing modulemultiple local indexing structures, one for each semantic category to reduce query response time significantly. In particular, the classification model combining linear discriminative mixture model (LDMM) and advanced score fusion scheme has been applied to estimate category of music accurately. Layered architecture enables EMIF to enjoy superior scalability and efficiency. To evaluate the approach, a set of experimental studies has been carried out using two large music test collections and the results demonstrate various advantages of EMIF over state-of-the-art approaches including efficiency, scalability and effectiveness.

show abstract

Joint Image-Text Hashing for Fast Large-Scale Cross-Media Retrieval Using Self-Supervised Deep Learning

Cited by 74 publications

References 31 publications

RGB-T Salient Object Detection via Fusing Multi-Level CNN Features

RGB-T Salient Object Detection via Fusing Multi-Level CNN Features

Deep Salient Object Detection With Contextual Information Guidance

Toward efficient indexing structure for scalable content-based music retrieval

Contact Info

Product

Resources

About