Learning Localized Perceptual Similarity Metrics for Interactive Categorization

Wah, Catherine; Maji, Subhransu; Belongie, Serge

doi:10.1109/wacv.2015.73

Cited by 9 publications

(5 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More broadly, our work is a contribution to the emerging research field at the intersection of human computation and computer vision to build hybrid systems that take advantage of the strengths of humans and computers together. For example, hybrid systems combine non-expert and algorithm strengths to perform the challenging fine-grained bird classification task typically performed by experts [8,43]. Another system decides how much human effort to allocate per image in order to segment the diversity of plausible foreground objects in a batch of images [21].…”

Section: Related Workmentioning

confidence: 99%

Predicting How to Distribute Work Between Algorithms and Humans to Segment an Image Batch

et al. 2019

View full text Add to dashboard Cite

Foreground object segmentation is a critical step for many image analysis tasks. While automated methods can produce high-quality results, their failures disappoint users in need of practical solutions. We propose a resource allocation framework for predicting how best to allocate a fixed budget of human annotation effort in order to collect higher quality segmentations for a given batch of images and automated methods. The framework is based on a prediction module that estimates the quality of given algorithm-drawn segmentations. We demonstrate the value of the framework for two novel tasks related to predicting how to distribute annotation efforts between algorithms and humans. Specifically, we develop two systems that automatically decide, for a batch of images, when to recruit humans versus computers to create 1) coarse segmentations required to initialize segmentation tools and 2) final, fine-grained segmentations. Experiments demonstrate the advantage of relying on a mix of human and computer efforts over relying on either resource alone for segmenting objects in images coming from three diverse modalities (visible, phase contrast microscopy, and fluorescence microscopy).

show abstract

Section: Related Workmentioning

confidence: 99%

Predicting How to Distribute Work Between Algorithms and Humans to Segment an Image Batch

et al. 2019

View full text Add to dashboard Cite

show abstract

“…, S T are available. This can be obtained by asking annotators to focus on a specific aspect when making pair-wise comparisons as in human in the loop tasks (Wah et al 2014;Wah, Maji, and Belongie 2015). Alternatively, different measures of similarity can come from considering multiple related metric learning problems as in (Parameswaran and Weinberger 2010;Rai, Lian, and Carin 2014).…”

Section: Jointly Learning Multiple Metricsmentioning

confidence: 99%

“…In addition to the triplet generalization error, we evaluated the embeddings in a classification task using a biological taxonomy of the bird species. Note that in Wah et al (2015) embeddings were used to interactively categorize images; here we simplify this process to enable detailed comparisons. We manually grouped the 200 classes to get 6 super classes so that the number of objects in all classes were balanced.…”

Section: Cub-200 Birds Datamentioning

confidence: 99%

“…A better approach would be to tell the annotator the desired view or the perspective of the object to use for measuring similarity. Such view-specific comparisons are not only easier for annotators, but they can also enable precise feedback for human "in the loop" tasks, such as, interactive fine-grained recognition (Wah, Maji, and Belongie 2015), thereby reducing the human effort. The main drawback of learning view specific embeddings independently is that the number of similarity comparisons scales linearly with the number of views.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Jointly Learning Multiple Measures of Similarities from Triplet Comparisons

Zhang¹,

Maji²,

Tomioka³

2015

Preprint

Self Cite

View full text Add to dashboard Cite

Similarity between objects is multi-faceted and it can be easier for human annotators to measure it when the focus is on a specific aspect. We consider the problem of mapping objects into view-specific embeddings where the distance between them is consistent with the similarity comparisons of the form "from the t-th view, object A is more similar to B than to C". Our framework jointly learns view-specific embeddings exploiting correlations between views. Experiments on a number of datasets, including one of multi-view crowdsourced comparison on bird images, show the proposed method achieves lower triplet generalization error when compared to both learning embeddings independently for each view and all views pooled into one view. Our method can also be used to learn multiple measures of similarity over input features taking class labels into account and compares favorably to existing approaches for multi-task metric learning on the ISOLET dataset.

show abstract

“…Many challenging machine‐learning problems involve humans in the loop (e.g., Branson, Van Horn, Wah, Perona, & Belongie, ; Deng, Krause, & Fei‐Fei, ; Jia, Abbott, Austerweil, Griffiths, & Darrell, ; Wah, Maji, & Belongie, ). Individuals may provide data to machine learning systems in the form of ratings, preferences, judgments, and labels.…”

Section: Introductionmentioning

confidence: 99%

Improving Human‐Machine Cooperative Classification Via Cognitive Theories of Similarity

Roads

Mozer

2016

Cognitive Science

View full text Add to dashboard Cite

Acquiring perceptual expertise is slow and effortful. However, untrained novices can accurately make difficult classification decisions (e.g., skin-lesion diagnosis) by reformulating the task as similarity judgment. Given a query image and a set of reference images, individuals are asked to select the best matching reference. When references are suitably chosen, the procedure yields an implicit classification of the query image. To optimize reference selection, we develop and evaluate a predictive model of similarity-based choice. The model builds on existing psychological literature and accommodates stochastic, dynamic shifts of attention among visual feature dimensions. We perform a series of human experiments with two stimulus types (rectangles, faces) and nine classification tasks to validate the model and to demonstrate the model's potential to boost performance. Our system achieves high accuracy for participants who are naive as to the classification task, even when the classification task switches from trial to trial.

show abstract

Learning Localized Perceptual Similarity Metrics for Interactive Categorization

Cited by 9 publications

References 50 publications

Predicting How to Distribute Work Between Algorithms and Humans to Segment an Image Batch

Predicting How to Distribute Work Between Algorithms and Humans to Segment an Image Batch

Jointly Learning Multiple Measures of Similarities from Triplet Comparisons

Improving Human‐Machine Cooperative Classification Via Cognitive Theories of Similarity

Contact Info

Product

Resources

About