Labeling images with a computer game

Ahn, Luis von; Dabbish, Laura

doi:10.1145/985692.985733

Cited by 1,557 publications

(1,033 citation statements)

References 3 publications

Supporting

Mentioning

997

Contrasting

Unclassified

Order By: Relevance

“…Namely, researchers have begun to "crowd source" experiments to obtain large amounts of data from many people in short order. Building off ingenious ideas, such as Luis von Ahn's "ESP Game" (a "game" that was the basis for how Google matches words to images; von Ahn & Dabbish, 2004), researchers have turned to outlets such as Amazon's Mechanical Turk (e.g., Buhrmester, Kwang, & Gosling, 2011) to rapidly distribute an experiment to many different participants.…”

Section: Drivers Of Prediction Accuracy In World Politicsmentioning

confidence: 99%

The psychology of intelligence analysis: Drivers of prediction accuracy in world politics.

Mellers¹,

Stone²,

Atanasov³

et al. 2015

Journal of Experimental Psychology: Applied

134

130

View full text Add to dashboard Cite

This article extends psychological methods and concepts into a domain that is as profoundly consequential as it is poorly understood: intelligence analysis. We report findings from a geopolitical forecasting tournament that assessed the accuracy of more than 150,000 forecasts of 743 participants on 199 events occurring over 2 years. Participants were above average in intelligence and political knowledge relative to the general population. Individual differences in performance emerged, and forecasting skills were surprisingly consistent over time. Key predictors were (a) dispositional variables of cognitive ability, political knowledge, and openmindedness; (b) situational variables of training in probabilistic reasoning and participation in collaborative teams that shared information and discussed rationales (Mellers, Ungar, et al., 2014); and (c) behavioral variables of deliberation time and frequency of belief updating. We developed a profile of the best forecasters; they were better at inductive reasoning, pattern detection, cognitive flexibility, and open-mindedness. They had greater understanding of geopolitics, training in probabilistic reasoning, and opportunities to succeed in cognitively enriched team environments. Last but not least, they viewed forecasting as a skill that required deliberate practice, sustained effort, and constant monitoring of current affairs. A t APA, we understand how challenging it can be for you to stay abreast of the latest and best academic research on psychology. Disciplines Finance and Financial Management | Social and Behavioral Sciences

show abstract

Section: Drivers Of Prediction Accuracy In World Politicsmentioning

confidence: 99%

The psychology of intelligence analysis: Drivers of prediction accuracy in world politics.

Mellers¹,

Stone²,

Atanasov³

et al. 2015

Journal of Experimental Psychology: Applied

134

130

View full text Add to dashboard Cite

show abstract

“…The ESP game, by Ahn & Dabbish [12], presents the same image to two players who cannot communicate. Their task is to produce the same word in as few tries as possible.…”

Section: Previous Workmentioning

confidence: 99%

Some Objects Are More Equal Than Others: Measuring and Predicting Importance

Spain

Perona

2008

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. We observe that everyday images contain dozens of objects, and that humans, in describing these images, give different priority to these objects. We argue that a goal of visual recognition is, therefore, not only to detect and classify objects but also to associate with each a level of priority which we call 'importance'. We propose a definition of importance and show how this may be estimated reliably from data harvested from human observers. We conclude by showing that a first-order estimate of importance may be computed from a number of simple image region measurements and does not require access to image meaning.

show abstract

“…These are Corel5k [4], ESP Game [20] and IAPRTC-12 [7]. While Corel-5k has become the de-facto dataset in this domain, the other two datasets are very challenging with significant diversity among their samples.…”

Section: Datasets and Featuresmentioning

confidence: 99%

“…Among the image annotation models being proposed in the past, generative or nearestneighbour (NN)-based models [5,8,11,19,23] have particularly been shown to be successful for large vocabulary datasets such as Corel-5k [4], ESP Game [20] and IAPRTC-12 [7]. The reason behind this is that in NN-based models, given a sample, the labels that are not present in the ground-truth of its neighbouring samples are simply ignored, rather than being considered as negative.…”

Section: Introductionmentioning

confidence: 99%

Exploring SVM for Image Annotation in Presence of Confusing Labels

Verma¹,

Jawahar²

2013

Procedings of the British Machine Vision Conference 2013

View full text Add to dashboard Cite

We address the problem of automatic image annotation in large vocabulary datasets. In such datasets, for a given label, there could be several other labels that act as its confusing labels. Three possible factors for this are (i) incomplete-labeling ("cars" vs. "vehicle"), (ii) label-ambiguity ("flowers" vs. "blooms"), and (iii) structural-overlap ("lion" vs. "tiger"). While previous studies in this domain have mostly focused on nearest-neighbour based models, we show that even the conventional one-vs-rest SVM significantly outperforms several benchmark models. We also demonstrate that with a simple modification in the hinge-loss of SVM, it is possible to significantly improve its performance. In particular, we introduce a tolerance-parameter in the hinge-loss. This makes the new model more tolerant against the errors in the classification of samples tagged with confusing labels as compared to other samples. This tolerance parameter is automatically determined using visual similarity and dataset statistics. Experimental evaluations demonstrate that our method (referred to as SVM with Variable Tolerance or SVM-VT) shows promising results on the task of image annotation on three challenging datasets, and establishes a baseline for such models in this domain.

show abstract

Labeling images with a computer game

Cited by 1,557 publications

References 3 publications

The psychology of intelligence analysis: Drivers of prediction accuracy in world politics.

The psychology of intelligence analysis: Drivers of prediction accuracy in world politics.

Some Objects Are More Equal Than Others: Measuring and Predicting Importance

Exploring SVM for Image Annotation in Presence of Confusing Labels

Contact Info

Product

Resources

About