Modeling continuous visual features for semantic image annotation and retrieval

Li, Zhixin; Shi, Zhiping; Liu, Xi; Shi, Zhongzhi

doi:10.1016/j.patrec.2010.11.015

Cited by 25 publications

(19 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Three alternatives of pLSA-based models that provided in [13] were presented by using asymmetric learning for semantic indexing of large image collections. Then, a Gaussian-multinomial pLSA (GM-pLSA) model [27] was presented to learn multimodal correlations from the image data by applying continuous feature vectors. Furthermore, the work in [28] extended pLSA to a higher-order formalism, so as to become applicable for more than two observable variables.…”

Section: Topic Models For Image Annotationmentioning

confidence: 99%

“…Therefore, Algorithm 2 is convergent. When coping with large-scale data (i.e., N K, C, D), the complexity of our modeling system is approximately linear to the number of images, which is much effective by comparing with the typical quadratic annotation models (e.g., pLSA [13] that requires O N 2 KC operations, GM-pLSA [27] that yields the number of operations roughly on O N 2 K 2 C , and so on). …”

Section: Parameter Estimation Via Variational Inferencementioning

confidence: 99%

“…For all images in the two standard datasets, our methods are compared with several most relevant and state-of-theart methods, including TagProp [3], pLSA [13], GM-pLSA [27], GM-LDA [29], Corr-LDA [29], topic regression multimodal latent Dirichlet allocation (tr-mmLDA) [30], and css-LDA [16].…”

Section: Evaluation Criteria and Baselinesmentioning

confidence: 99%

See 2 more Smart Citations

Class-specific Gaussian-multinomial latent Dirichlet allocation for image annotation

Qian

Zhong

Wang

2015

EURASIP J. Adv. Signal Process.

View full text Add to dashboard Cite

Image annotation has been a challenging problem due to the well-known semantic gap between two heterogeneous information modalities, i.e., the visual modality referring to low-level visual features and the semantic modality referring to high-level human concepts. To bridge the semantic gap, we present an extension of latent Dirichlet allocation (LDA), denoted as class-specific Gaussian-multinomial latent Dirichlet allocation (csGM-LDA), in an effort to simulate the human's visual perception system. An analysis of previous supervised LDA models shows that the topics discovered by generative LDA models are driven by general image regularities rather than the semantic regularities for image annotation. To address this, csGM-LDA is introduced by using class supervision at the level of visual features for multimodal topic modeling. The csGM-LDA model combines the labeling strength of topic supervision with the flexibility of topic discovery, and the modeling problem can be effectively solved by a variational expectation-maximization (EM) algorithm. Moreover, as natural images usually generate an enormous size of high-dimensional data in annotation applications, an efficient descriptor based on Laplacian regularized uncorrelated tensor representation is proposed for explicitly exploiting the manifold structures in the high-order image space. Experimental results on two standard annotation datasets have shown the effectiveness of the proposed method by comparing with several state-of-the-art annotation methods.

show abstract

Section: Topic Models For Image Annotationmentioning

confidence: 99%

Section: Parameter Estimation Via Variational Inferencementioning

confidence: 99%

See 1 more Smart Citation

Class-specific Gaussian-multinomial latent Dirichlet allocation for image annotation

Qian

Zhong

Wang

2015

EURASIP J. Adv. Signal Process.

View full text Add to dashboard Cite

show abstract

“…In our previous works [22][23][24], we propose PLSA-FUSION and GM-PLSA. PLSA-FUSION employs two PLSA models to capture semantic information from vis-ual and textual modalities respectively, while GM-PLSA improves the learning procedure by modeling visual features directly.…”

Section: Related Workmentioning

confidence: 99%

Combining Generative/Discriminative Learning for Automatic Image Annotation and Retrieval

Li¹,

Tang²,

Zhao³

et al. 2012

IJIS

Self Cite

View full text Add to dashboard Cite

In order to bridge the semantic gap exists in image retrieval, this paper propose an approach combining generative and discriminative learning to accomplish the task of automatic image annotation and retrieval. We firstly present continuous probabilistic latent semantic analysis (PLSA) to model continuous quantity. Furthermore, we propose a hybrid framework which employs continuous PLSA to model visual features of images in generative learning stage and uses ensembles of classifier chains to classify the multi-label data in discriminative learning stage. Since the framework combines the advantages of generative and discriminative learning, it can predict semantic annotation precisely for unseen images. Finally, we conduct a series of experiments on a standard Corel dataset. The experiment results show that our approach outperforms many state-of-the-art approaches.

show abstract

“…Recently, probabilistic Latent Semantic Analysis (pLSA) [1] known as a latent aspect model has been widely used in the multimedia domain [2][3][4][5][6]. In these applications, so as to fit pLSA, the continuous visual or audio features are required to be quantized into discrete words in advance (so in this paper standard pLSA is also called discrete pLSA).…”

Section: Introductionmentioning

confidence: 99%

Graph regularized GM-pLSA with application to video classification

Zhong

Zhang

2013

2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW)

View full text Add to dashboard Cite

Seeing that standard probabilistic Latent Semantic Analysis (pLSA) only handles discrete quantity, pLSA with Gaussian Mixtures (GM-pLSA) is proposed to extend it to continuous feature space by using Gaussian Mixture Model (GMM) to describe the feature distribution under each aspect. However, inheriting from standard pLSA, GM-pLSA still assumes that terms are independent and ignores the intrinsic correlations between them. In this paper, we present a graph regularized GM-pLSA (GRGM-pLSA) model as an extension of GM-pLSA to take advantage of this neglected term correlation information for performance improvement. Appealing to the manifold learning theory, such a useful clue is captured by a graph regularizer and embedded into the process of model learning. In the application of video classification, two kinds of term correlation respectively representing temporal consistency and visual similarity between sub-shots are evaluated. Experimental results show that our proposed GRGM-pLSA outperforms GM-pLSA.

show abstract

Modeling continuous visual features for semantic image annotation and retrieval

Cited by 25 publications

References 20 publications

Class-specific Gaussian-multinomial latent Dirichlet allocation for image annotation

Class-specific Gaussian-multinomial latent Dirichlet allocation for image annotation

Combining Generative/Discriminative Learning for Automatic Image Annotation and Retrieval

Graph regularized GM-pLSA with application to video classification

Contact Info

Product

Resources

About