2019
DOI: 10.1186/s13104-019-4764-4
|View full text |Cite
|
Sign up to set email alerts
|

Reliability of crowdsourcing as a method for collecting emotions labels on pictures

Abstract: ObjectiveIn this paper we study if and under what conditions crowdsourcing can be used as a reliable method for collecting high-quality emotion labels on pictures. To this end, we run a set of crowdsourcing experiments on the widely used IAPS dataset, using the Self-Assessment Manikin (SAM) emotion collection instrument, in order to rate pictures on valence, arousal and dominance, and explore the consistency of crowdsourced results across multiple runs (reliability) and the level of agreement with the gold lab… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
1
1

Citation Types

0
2
0

Year Published

2020
2020
2024
2024

Publication Types

Select...
6

Relationship

0
6

Authors

Journals

citations
Cited by 6 publications
(2 citation statements)
references
References 13 publications
(10 reference statements)
0
2
0
Order By: Relevance
“…SAM is a non-verbal, pictorial (figure-based) scale, with three items to assess the valence, arousal, and dominance associated with the reaction to an emotional experience. Previous research suggests that participants may experience difficulty understanding how to rate their emotional experience (particularly in the dominance dimension) based on the pictorial response scale (e.g., Korovina et al, 2019). To address this difficulty, we added a label for the lowest and highest point of each dimension.…”
Section: Emotional Experiencementioning
confidence: 99%
“…SAM is a non-verbal, pictorial (figure-based) scale, with three items to assess the valence, arousal, and dominance associated with the reaction to an emotional experience. Previous research suggests that participants may experience difficulty understanding how to rate their emotional experience (particularly in the dominance dimension) based on the pictorial response scale (e.g., Korovina et al, 2019). To address this difficulty, we added a label for the lowest and highest point of each dimension.…”
Section: Emotional Experiencementioning
confidence: 99%
“…The abundance of contents available online, such as TV-shows and movies, allows researchers to glean rich emotion data representative of various contexts efficiently. Crowdsourcing further supports inexpensive data annotation while serving as another data source 40 , 41 . Datasets of this type have advantages in sample size and the diversity of subjects, but generalizability remains an issue.…”
Section: Background and Summarymentioning
confidence: 99%