Semantic Consistency Cross-Modal Retrieval With Semi-Supervised Graph Regularization

Xu, Gongwen; Li, Xiaomei; Zhang, Zhijun

doi:10.1109/access.2020.2966220

Cited by 17 publications

(14 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, the representation learning methods of deep learning have attracted extensive attention [28][29][30][31][32][33][34]. e common models of knowledge graph representation learning include distance model, energy model, matrix decomposition model, bilinear model, translation model, and so on [35].…”

Section: Recommendation Methods Based On Knowledge Graphmentioning

confidence: 99%

Personalized Course Recommendation System Fusing with Knowledge Graph and Collaborative Filtering

Jia

Shi

et al. 2021

Computational Intelligence and Neuroscience

Self Cite

View full text Add to dashboard Cite

Personalized courses recommendation technology is one of the hotspots in online education field. A good recommendation algorithm can stimulate learners’ enthusiasm and give full play to different learners’ learning personality. At present, the popular collaborative filtering algorithm ignores the semantic relationship between recommendation items, resulting in unsatisfactory recommendation results. In this paper, an algorithm combining knowledge graph and collaborative filtering is proposed. Firstly, the knowledge graph representation learning method is used to embed the semantic information of the items into a low-dimensional semantic space; then, the semantic similarity between the recommended items is calculated, and then, this item semantic information is fused into the collaborative filtering recommendation algorithm. This algorithm increases the performance of recommendation at the semantic level. The results show that the proposed algorithm can effectively recommend courses for learners and has higher values on precision, recall, and F1 than the traditional recommendation algorithm.

show abstract

Section: Recommendation Methods Based On Knowledge Graphmentioning

confidence: 99%

Personalized Course Recommendation System Fusing with Knowledge Graph and Collaborative Filtering

Jia

Shi

et al. 2021

Computational Intelligence and Neuroscience

Self Cite

View full text Add to dashboard Cite

show abstract

“…S is the semantic matrix of image and text, X and Y represents the feature matrices of image and text respectively, λ, α, β 1 and β 2 are balance parameters. A semantic consistency cross-modal retrieval with semi-supervised graph regularization (SCCMR) method is introduced in [66] which ensures a globally optimal solution by merging prediction of labels and optimization of projection matrices to a unified architecture. Simultaneously, the method also considers nearest neighbors in potential image-text subspace and image-text with the same semantics using graph embedding.…”

Section: Subspace Learningmentioning

confidence: 99%

Comparative analysis on cross-modal information retrieval: A review

Kaur

Pannu

Malhi

2021

Computer Science Review

View full text Add to dashboard Cite

“…Li et al [55] proposed a reasoning model based on graph convolutional networks to enhanced visual representations by region relationship reasoning and global semantic reasoning. Xu et al [51] applied graph embedding method to ensure the approximation of paired images and texts in joint embedding space. Similar to the above approaches, we lay emphasis on taking image-text matching as the representations learning procedure for cross-modal data.…”

Section: Related Work a Image-text Matchingmentioning

confidence: 99%

Multi-Modal Memory Enhancement Attention Network for Image-Text Matching

Lin

Wang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Image-text matching is an attractive research topic in the community of vision and language. The key element to narrow the ''heterogeneity gap'' between visual and textual data lies in how to learn powerful and robust representations for both modalities. This paper proposes to alleviate this issue to achieve the fine-grained visual-textual alignment from two aspects: exploiting attention mechanism to locate the semantically meaningful portion and leveraging the memory network to capture the long-term contextual knowledge. Unlike most existing studies sorely focus on exploring the cross-modal associations at the fragment level, our designed Collaborative Dual Attention (CDA) module is able to model the semantic interdependencies from both perspectives of fragment and channel. Furthermore, considering the usage of long-term contextual knowledge contributes to compensate for detailed semantics concealed in the rarely appeared image-text pairs, we present to learn the joint representations by constructing a Multi-Modal Memory Enhancement (M3E) module. Specifically, it sequentially restores the intra-modal and multi-modal information into the memory items, and they conversely persistently memorize cross-modal shared semantics to improve the latent embeddings. By incorporating both CDA and M3E modules into a deep architecture, our approach generates more semantically consistent embeddings for representing images and texts. Extensive experiments demonstrate our model can achieve the state-of-the-art results on two public benchmark datasets. INDEX TERMS Image-text matching, attention mechanism, memory network.

show abstract

Semantic Consistency Cross-Modal Retrieval With Semi-Supervised Graph Regularization

Cited by 17 publications

References 38 publications

Personalized Course Recommendation System Fusing with Knowledge Graph and Collaborative Filtering

Personalized Course Recommendation System Fusing with Knowledge Graph and Collaborative Filtering

Comparative analysis on cross-modal information retrieval: A review

Multi-Modal Memory Enhancement Attention Network for Image-Text Matching

Contact Info

Product

Resources

About