Exploring the Use of Deep Learning with Crowdsourcing to Annotate Images

Anjum, Samreen; Verma, Ambika; Dang, Brandon; Gurari, Danna

doi:10.15346/hc.v8i2.121

Cited by 8 publications

(6 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Alerts: timeliness is the primary goal of this analysis, that aims at generating early alerts for a starting event. In this case, the balance between the manual and automatic activities to be performed must be assessed, as the results are useful if they are delivered in the first hours after the event onset [10].…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

A Citizen Science Approach for Analyzing Social Media With Crowdsourcing

et al. 2023

View full text Add to dashboard Cite

Social media have the potential to provide timely information about emergency situations and sudden events. However, finding relevant information among the millions of posts being added every day can be difficult, and in current approaches developing an automatic data analysis project requires time and technical skills. This work presents a new approach for the analysis of social media posts, based on configurable automatic classification combined with Citizen Science methodologies. The process is facilitated by a set of flexible, automatic and open-source data processing tools called the Citizen Science Solution Kit. The kit provides a comprehensive set of tools that can be used and personalized in different situations, particularly during natural emergencies, starting from images and text contained in the posts. The tools can be employed by citizen scientists for filtering, classifying, and geolocating the content with a human-in-the-loop approach to support the data analyst, including feedback and suggestions on how to configure the automated tools, and techniques to gather inputs from citizens. Using flooding scenario as a guiding example, this paper illustrates the structure and functioning of the different tools proposed to support citizens scientists in their projects, and a methodological approach to their use. The process is then validated by discussing three case studies based on the Albania earthquake of 2019, the Covid-19 pandemic, and the Thailand floods of 2021. The results suggest that a flexible approach to tools composition and configuration can support a timely setup of an analysis project by citizen scientists, especially in case of emergencies in unexpected locations.

show abstract

Section: Resultsmentioning

confidence: 99%

“…For automatic analysis of images, many recent approaches are based on AI and, in particular, neural networks [8], [9]. Hybrid deeplearning and crowdsourcing approaches were analyzed as well [10].…”

Section: Related Workmentioning

confidence: 99%

A Citizen Science Approach for Analyzing Social Media With Crowdsourcing

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Therefore, manual image annotation process has (r)evolved into a new generation, namely human-centered/crowdsourcing-based image interpretation and understanding. Following this (relatively) novel and working idea, many proposals for image annotation based on human intelligence have been introduced (Anjum et al, 2021;Ho et al, 2010;Liu et al, 2018;Rashtchian et al, 2010).…”

Section: Image Annotationmentioning

confidence: 99%

CAPTCHA for crowdsourced image annotation: directions and efficiency analysis

Moradi

Keyvanpour

2022

AJIM

View full text Add to dashboard Cite

Purpose Image annotation plays an important role in image retrieval process, especially when it comes to content-based image retrieval. In order to compensate the intrinsic weakness of machines in performing cognitive task of (human-like) image annotation, leveraging humans’ knowledge and abilities in the form of crowdsourcing-based annotation have gained momentum. Among various approaches for this purpose, an innovative one is integrating the annotation process into the CAPTCHA workflow. In this paper, the current state of the research works in the field and experimental efficiency analysis of this approach are investigated.Design/methodology/approach At first, and with the aim of presenting a current state report of research studies in the field, a comprehensive literature review is provided. Then, several experiments and statistical analyses are conducted to investigate how CAPTCHA-based image annotation is reliable, accurate and efficient.Findings In addition to study of current trends and best practices for CAPTCHA-based image annotation, the experimental results demonstrated that despite some intrinsic limitations on leveraging the CAPTCHA as a crowdsourcing platform, when the challenge, i.e. annotation task, is selected and designed appropriately, the efficiency of CAPTCHA-based image annotation can outperform traditional approaches. Nonetheless, there are several design considerations that should be taken into account when the CAPTCHA is used as an image annotation platform.Originality/value To the best of the authors’ knowledge, this is the first study to analyze different aspects of the titular topic through exploration of the literature and experimental investigation. Therefore, it is anticipated that the outcomes of this study can draw a roadmap for not only CAPTCHA-based image annotation but also CAPTCHA-mediated crowdsourcing and even image annotation.

show abstract

“…-Alerts: early alerts for a starting event are generated, and timeliness is the primary goal of the analysis. In this case, the balance between the manual and automatic activities to be performed must be assessed, as the results are useful if they are delivered in the first hours after the event onset [3].…”

Section: Motivating Scenariomentioning

confidence: 99%

“…For automatic analysis of images, many recent approaches are based on AI and in particular neural networks, for example, [24,32,35]. Hybrid deep-learning and crowdsourcing approaches were analyzed in [3]. The opportunities and challenges of using social media in emergencies were analyzed in [48].…”

Section: Related Workmentioning

confidence: 99%

Analyzing social media with crowdsourcing in Crowd4SDG

Bono¹,

Mülâyim²,

Cappiello³

et al. 2022

Preprint

View full text Add to dashboard Cite

Social media have the potential to provide timely information about emergency situations and sudden events. However, finding relevant information among millions of posts being posted every day can be difficult, and developing a data analysis project usually requires time and technical skills. This study presents an approach that provides flexible support for analyzing social media, particularly during emergencies. Different use cases in which social media analysis can be adopted are introduced, and the challenges of retrieving information from large sets of posts are discussed. The focus is on analyzing images and text contained in social media posts and a set of automatic data processing tools for filtering, classification, and geolocation of content with a human-in-the-loop approach to support the data analyst. Such support includes both feedback and suggestions to configure automated tools, and crowdsourcing to gather inputs from citizens. The results are validated by discussing three case studies developed within the Crowd4SDG H2020 European project.

show abstract

Exploring the Use of Deep Learning with Crowdsourcing to Annotate Images

Cited by 8 publications

References 50 publications

A Citizen Science Approach for Analyzing Social Media With Crowdsourcing

A Citizen Science Approach for Analyzing Social Media With Crowdsourcing

CAPTCHA for crowdsourced image annotation: directions and efficiency analysis

Analyzing social media with crowdsourcing in Crowd4SDG

Contact Info

Product

Resources

About