Multiple deep neural networks with multiple labels for cross-modal hashing retrieval

Xie, Yicai; Zeng, Xianhua; Wang, Tinghua; Xu, Liming; Wang, Dingjie

doi:10.1016/j.engappai.2022.105090

Cited by 11 publications

(7 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, even between instance pairs with the same similarity, they should be different, not identical. Multiple deep neural networks with multiple labels for cross-modal hashing retrieval (MDMCH) [2] to measure the difference between instances by calculating semantic factors.…”

Section: B Multi-label Methodsmentioning

confidence: 99%

“…Inspired by MDMCH [2], we use multi-label non-cooccurrence information to enhance the similarity matrix, which can make our similarity matrix more delicate. Moreover, we introduce Graph Convolutional Network to improve our representation of text features.…”

Section: Graph Convolutional Networkmentioning

confidence: 99%

“…Through the above method, we can see that the similarity between instances is no longer a simple 0 and 1, but a number between 0 and 1, which helps us to distinguish the degree of similarity between different instances. For us, inspired by MDMCH [2], we use semantic factors as multi-label non-co-occurrence information to enhance the similarity matrix, which can make our similarity matrix more delicate.…”

Section: B Similarity Constructionmentioning

confidence: 99%

“…We have compared our methods with the state-of-the-art existing methods, including CMSSH [19], SePH [20], SCM [29] 0.5445 0.5597 0.5803 0.7316 0.7343 0.7446 0.4536 0.4727 0.4919 CMHH [32] 0.5530 0.5698 0.5559 0.7334 0.7281 0.7444 0.4903 0.5074 0.5152 CHN [33] 0.5754 0.5966 0.6015 0.7504 0.7495 0.7461 0.4962 0.5070 0.5241 SSAH [36] 0.6163 0.6278 0.6140 0.7745 0.7882 0.7990 0.5348 0.5619 0.5781 SCAHN [38] 0.6429 0.6510 0.6635 0.8168 0.8311 0.8328 0.5427 0.5764 0.5908 MLSPH [3] 0.6405 0.6604 0.6734 0.8076 0.8235 0.8337 0.5342 0.5721 0.5994 MDMCH [2] 0 [21], GSPH [22], DCMH [29], PRDH [31], CMHH [32], CHN [? ], SSAH [36], SCAHN [38], MLSPH [3], MDMCH [2] (Because we cannot obtain the complete experimental code of MDMCH, we cannot conduct a complete experiment on IAPRTC-12 dataset). From left to right, there are four hand-crafted methods and eight methods based on deep features.…”

Section: Model Evaluationmentioning

confidence: 99%

“…However, even if the number of co-occurrence labels between two pairs of instances is equal, their similarity should be different. Inspired by MDMCH [2], we add multi-label non-co-occurrence information between instances to multilabel similarity measurement. If instances A and B have the same number of shared labels as instances A and C, but the number of non-co-occurrence labels between instances A and C is less than the instances A and B, then we have reason to think that the similarity of the latter is greater than the former (Figure 2).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Non-Co-Occurrence Enhanced Multi-Label Cross-Modal Hashing Retrieval Based on Graph Convolutional Network

Fan

Lin

2023

IEEE Access

View full text Add to dashboard Cite

Supervised cross-modal retrieval has significant advantages in retrieval efficiency and storage cost. In the field of hashing retrieval, existing supervised methods are divided into single-label and multilabel methods. For the single-label method, simply using a single label to measure the semantic relevance between instances will cause an error in supervision information. However, the existing multi-label hashing methods also have some problems. For example, only considering the co-occurrence of multiple labels among instances may not accurately reflect their similarity. At the same time, in the previous methods, the text modality processing did not reach the fine level of image modality, resulting in insufficient use of text information. We proposed Non-co-occurrence enhanced Multi-label cross-modal hashing retrieval based on Graph Convolutional Network (MHGCN) to address these issues. Firstly, we introduced a multi-label non-co-occurrence similarity measurement method, which adds multi-label non-co-occurrence information among instances in the multi-label similarity measurement to measure the differences between instances; Secondly, we used Graph Convolutional Networks (GCNS) to process the information on text modality; Thirdly, we introduced the memory mechanism to restrict the difference of hash code learning. Many experiments on three commonly used benchmark datasets show that the proposed Non-co-occurrence enhanced Multi-label cross-modal hashing retrieval based on Graph Convolutional Network (MHGCN) method has excellent performance.INDEX TERMS Non-co-occurrence enhanced hashing retrieval, Graph Convolutional Network, Multilabel method.

show abstract

Section: B Multi-label Methodsmentioning

confidence: 99%

Section: Graph Convolutional Networkmentioning

confidence: 99%

Section: B Similarity Constructionmentioning

confidence: 99%