Joint Image Clustering and Labeling by Matrix Factorization

Hong, Seunghoon; Choi, Jonghyun; Feyereisl, Jan; Han, Bohyung; Davis, Larry S.

doi:10.1109/tpami.2015.2487982

Cited by 41 publications

(16 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…High-level concepts in latent space can be further discovered in semi-or unsupervised manner. Reference [22] uses a limited reference database of images with attributes to cluster and annotate a set of unlabeled input images. By coupling a CNN with a set of data-dependent binary attributes, [18] seeks to automatically discover image attributes.…”

Section: A Concept Vectorsmentioning

confidence: 99%

Concept Saliency Maps to Visualize Relevant Features in Deep Generative Models

Brocki

Chung

2019

2019 18th IEEE International Conference on Machine Learning and Applications (ICMLA)

View full text Add to dashboard Cite

Evaluating, explaining, and visualizing high-level concepts in generative models, such as variational autoencoders (VAEs), is challenging in part due to a lack of known prediction classes that are required to generate saliency maps in supervised learning. While saliency maps may help identify relevant features (e.g., pixels) in the input for classification tasks of deep neural networks, similar frameworks are understudied in unsupervised learning. Therefore, we introduce a new method of obtaining saliency maps for latent representations of known or novel high-level concepts, often called concept vectors in generative models. Concept scores, analogous to class scores in classification tasks, are defined as dot products between concept vectors and encoded input data, which can be readily used to compute the gradients. The resulting concept saliency maps are shown to highlight input features deemed important for high-level concepts. Our method is applied to the VAE's latent space of CelebA dataset in which known attributes such as "smiles" and "hats" are used to elucidate relevant facial features. Furthermore, our application to spatial transcriptomic (ST) data of a mouse olfactory bulb demonstrates the potential of latent representations of morphological layers and molecular features in advancing our understanding of complex biological systems. By extending the popular method of saliency maps to generative models, the proposed concept saliency maps help improve interpretability of latent variable models in deep learning.Codes to reproduce and to implement concept saliency maps: https://github.com/lenbrocki/concept-saliency-maps

show abstract

Section: A Concept Vectorsmentioning

confidence: 99%

Concept Saliency Maps to Visualize Relevant Features in Deep Generative Models

Brocki

Chung

2019

2019 18th IEEE International Conference on Machine Learning and Applications (ICMLA)

View full text Add to dashboard Cite

show abstract

“…e CDONMTF model has two types of parameters: decomposition dimension values p and q; regularization parameters α, β, and λ. Although many literature studies have studied how to choose the dimension [32,46,47] and regularization parameters [7,48,49], there is no unique way to select these parameters. For the choice of p and q, we use the following methods [19]:…”

Section: Parameter Analysismentioning

confidence: 99%

Constrained Dual Graph Regularized Orthogonal Nonnegative Matrix Tri-Factorization for Co-Clustering

Luo

2019

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Coclustering approaches for grouping data points and features have recently been receiving extensive attention. In this paper, we propose a constrained dual graph regularized orthogonal nonnegative matrix trifactorization (CDONMTF) algorithm to solve the coclustering problems. The new method improves the clustering performance obviously by employing hard constraints to retain the priori label information of samples, establishing two nearest neighbor graphs to encode the geometric structure of data manifold and feature manifold, and combining with biorthogonal constraints as well. In addition, we have also derived the iterative optimization scheme of CDONMTF and proved its convergence. Clustering experiments on 5 UCI machine-learning data sets and 7 image benchmark data sets show that the achievement of the proposed algorithm is superior to that of some existing clustering algorithms.

show abstract

“…With the help of deep Convolutional Neural Networks (CNNs), they train to output a set of discriminative, binary attributes often with semantic meanings. Hong et al [34] propose a novel algorithm to cluster and annotate a set of input images with semantic concepts jointly. They employ non-negative matrix factorization with sparsity and orthogonality con-straints to learn the label-based representations with the side information (a labeled reference image database) obtaining promising results.…”

Section: Related Workmentioning

confidence: 99%

What is the best way for extracting meaningful attributes from pictures?

Liu

Wiliem

Chen

et al. 2017

Pattern Recognition

View full text Add to dashboard Cite

Automatic attribute discovery methods have gained in popularity to extract sets of visual attributes from images or videos for various tasks. Despite their good performance in some classification tasks, it is difficult to evaluate whether the attributes discovered by these methods are meaningful and which methods are the most appropriate to discover attributes for visual descriptions. In its simplest form, such an evaluation can be performed by manually verifying whether there is any consistent identifiable visual concept distinguishing between positive and negative exemplars labelled by an attribute. This manual checking is tedious, expensive and labour intensive. In addition, comparisons between different methods could also be problematic as it is not clear how one could quantitatively decide which attribute is more meaningful than the others. In this paper, we propose a novel attribute meaningfulness metric to address this challenging problem. With this metric, automatic quantitative evaluation can be performed on the attribute sets; thus, reducing the enormous effort to perform manual evaluation. The proposed metric is applied to some recent automatic attribute discovery and hashing methods on four attribute-labelled datasets. To further validate the efficacy of the proposed method, we conducted a user study. In addition, we also compared our metric with a semi-supervised attribute discover method using the mixture of probabilistic PCA. In our evaluation, we gleaned several insights that could be beneficial in developing new automatic attribute discovery methods.

show abstract

Joint Image Clustering and Labeling by Matrix Factorization

Cited by 41 publications

References 47 publications

Concept Saliency Maps to Visualize Relevant Features in Deep Generative Models

Concept Saliency Maps to Visualize Relevant Features in Deep Generative Models

Constrained Dual Graph Regularized Orthogonal Nonnegative Matrix Tri-Factorization for Co-Clustering

What is the best way for extracting meaningful attributes from pictures?

Contact Info

Product

Resources

About