Weakly Supervised Visual Dictionary Learning by Harnessing Image Attributes

Abstract

Abstract-Bag-of-features (BoFs) representation has been extensively applied to deal with various computer vision applications. To extract discriminative and descriptive BoF, one important step is to learn a good dictionary to minimize the quantization loss between local features and codewords. While most existing visual dictionary learning approaches are engaged with unsupervised feature quantization, the latest trend has turned to supervised learning by harnessing the semantic labels of images or regions. However, such labels are typically too expensive to acquire, which restricts the scalability of supervised dictionary learning approaches. In this paper, we propose to leverage image attributes to weakly supervise the dictionary learning procedure without requiring any actual labels. As a key contribution, our approach establishes a generative hidden Markov random field (HMRF), which models the quantized codewords as the observed states and the image attributes as the hidden states, respectively. Dictionary learning is then performed by supervised grouping the observed states, where the supervised information is stemmed from the hidden states of the HMRF. In such a way, the proposed dictionary learning approach incorporates the image attributes to learn a semanticpreserving BoF representation without any genuine supervision. Experiments in large-scale image retrieval and classification tasks corroborate that our approach significantly outperforms the state-of-the-art unsupervised dictionary learning approaches.Index Terms-Bag-of-features, visual dictionary, image attribute, weakly supervised learning, hidden Markov random field, image classification, image search.

Block-Row Sparse Multiview Multilabel Learning for Image Classification

“…The regularizer (including the 2,1 -norm regularizer [57], [58], the 2 2,1 -norm regularizer [52], and the 1,∞ -norm regularizer [59]) is often used for the application of joint sparse learning. For example, the 2,1 -norm regularizer penalizes all coefficients in a whole row for jointly generating the row sparsity, please see Fig.…”

“…For example, the global feature has the ability to generalize the whole image with only a single vector so that its use in image classification is straightforward, while the local feature is calculated at multiple points (or regions) of an image so that the representation is robust to occlusion and clutter [8]- [10]. Studies have shown that only single visual feature is not always robust to all types of scenarios [11]- [15].…”

Content based medical image retrieval using dictionary learning

“…al. [20] proposed a novel weakly unsupervised DL method. They incorporated cheaply available visual attributes very effectively into dictionary learning and demonstrated the efficacy, scalability of the method in large scale image retrieval and classification tasks.…”

“…This is because it is very difficult to define suitable attributes from medical databases, which is in contrast to the databases of general images where attributes such as tree, car and feather etc can work well. Inspired by the ideas in [20], our future efforts shall attempt to identify and incorporate visual attributes into DL to improve upon the performance of CBMIR. In addition, as medical images come with different transformations (such as scaling), our future work shall further aim at addressing the invariant CBMIR with respect to other transformations as well.…”

Kinship verification in multi-linear coherent spaces

“…Although image analysis and retrieval have been investigated for decades [16,17,39], it is still a challenging task to explore the kinship information behind face images. Recently, the abundance of face images has attracted researchers' interests to explore kinship relations based on facial appearance [21, 28-30, 32, 40, 41].…”