Video search reranking through random walk over document-level context graph

Hsu, Winston H.; Kennedy, Lyndon; Chang, Shih‐Fu

doi:10.1145/1291233.1291446

Cited by 199 publications

(195 citation statements)

References 17 publications

Supporting

Mentioning

192

Contrasting

Order By: Relevance

“…Specifically, the random-walk approach for multimodal fusion was introduced in [12], where the fusion of textual and visual information leads to improved performance in the video search task. The framework in [2] includes as special cases all well-known fusion models (e.g.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Multimedia retrieval based on non-linear graph-based fusion and partial least squares regression

Gialampoukidis

Moumtzidou

Liparas

et al. 2017

Multimed Tools Appl

View full text Add to dashboard Cite

Heterogeneous sources of information, such as images, videos, text and metadata are often used to describe different or complementary views of the same multimedia object, especially in the online news domain and in large annotated image collections. The retrieval of multimedia objects, given a multimodal query, requires the combination of several sources of information in an efficient and scalable way. Towards this direction, we provide a novel unsupervised framework for multimodal fusion of visual and textual similarities, which are based on visual features, visual concepts and textual metadata, integrating non-linear graph-based fusion and Partial Least Squares Regression. The fusion strategy is based on the construction of a multimodal contextual similarity matrix and the non-linear combination of relevance scores from query-based similarity vectors. Our framework can employ more than two modalities and high-level information, without increase in memory complexity, when compared to state-of-the-art baseline methods. The experimental comparison is done in three public multimedia collections in the multimedia retrieval task. The results have shown that the proposed method outperforms the baseline methods, in terms of Mean Average Precision and Precision@20.

show abstract

Section: Related Workmentioning

confidence: 99%

“…The transition probability p κλ between two multimodal items is also depicted in Figure 2 and provides the weight on the link from node κ to node λ. The graphbased approach has been proposed in [12] in the context of video retrieval, but is directly applicable to any pairs of modalities.…”

Section: Graph-based Fusion In Multimedia Retrievalmentioning

confidence: 99%

Multimedia retrieval based on non-linear graph-based fusion and partial least squares regression

Gialampoukidis

Moumtzidou

Liparas

et al. 2017

Multimed Tools Appl

View full text Add to dashboard Cite

show abstract

“…The images reranking methods can be classified into that of classification based [12,25], clustering based [2,3,4,7,14], graph based [8,16], and learning to rerank [22,26,27].…”

Section: Related Workmentioning

confidence: 99%

“…The process of reranking is formulated as random walk over the graph and the relevance scores are propagated through the edges. So, the relevance of the image to the query keyword can be measured by relevance scores defined on the graphs [8,16].…”

Section: Related Workmentioning

confidence: 99%

An Adaptive Query Prototype Modeling Method for Image Search Reranking

Jiang

Yang

et al. 2013

2013 IEEE International Conference on Computer Vision Workshops

View full text Add to dashboard Cite

“…The other popular method is the graph based random walk, where the stationary probability of the random walk process is used as the ranking scores of the nodes. Such ranking method has been widely used in video search reranking [11], social media tag ranking [15], etc. However, all these methods are limited to homogeneous networks with nodes of the same type, which are inadequate for modeling the heterogeneous entities and relations.…”

Section: Related Workmentioning

confidence: 99%

Hybrid social media network

Liu

Chen

et al. 2012

Proceedings of the 20th ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

Analysis and recommendation of multimedia information can be greatly improved if we know the interactions between the content, user, and concept, which can be easily observed from the social media networks. However, there are many heterogeneous entities and relations in such networks, making it difficult to fully represent and exploit the diverse array of information. In this paper, we develop a hybrid social media network, through which the heterogeneous entities and relations are seamlessly integrated and a joint inference procedure across the heterogeneous entities and relations can be developed. The network can be used to generate personalized information recommendation in response to specific targets of interests, e.g., personalized multimedia albums, target advertisement and friend/topic recommendation. In the proposed network, each node denotes an entity and the multiple edges between nodes characterize the diverse relations between the entities (e.g., friends, similar contents, related concepts, favorites, tags, etc). Given a query from a user indicating his/her information needs, a propagation over the hybrid social media network is employed to infer the utility scores of all the entities in the network while learning the edge selection function to activate only a sparse subset of relevant edges, such that the query information can be best propagated along the activated paths. Driven by the intuition that much redundancy exists among the diverse relations, we have developed a robust optimization framework based on several sparsity principles. We show significant performance gains of the proposed method over the state of the art in multimedia retrieval and recommendation using data crawled from social media sites. To the best of our knowledge, this is the first model supporting not only aggregation but also judicious selection of heterogeneous relations in the social media networks.

show abstract

Video search reranking through random walk over document-level context graph

Cited by 199 publications

References 17 publications

Multimedia retrieval based on non-linear graph-based fusion and partial least squares regression

Multimedia retrieval based on non-linear graph-based fusion and partial least squares regression

An Adaptive Query Prototype Modeling Method for Image Search Reranking

Hybrid social media network

Contact Info

Product

Resources

About