GroundTruth: Augmenting Expert Image Geolocation with Crowdsourcing and Shared Representations

Abstract

Expert investigators bring advanced skills and deep experience to analyze visual evidence, but they face limits on their time and attention. In contrast, crowds of novices can be highly scalable and parallelizable, but lack expertise. In this paper, we introduce the concept of shared representations for crowd--augmented expert work, focusing on the complex sensemaking task of image geolocation performed by professional journalists and human rights investigators. We built GroundTruth, an online system that uses three shared representations-a diagram, grid, and heatmap-to allow experts to work with crowds in real time to geolocate images. Our mixed-methods evaluation with 11 experts and 567 crowd workers found that GroundTruth helped experts geolocate images, and revealed challenges and success strategies for expert-crowd interaction. We also discuss designing shared representations for visual search, sensemaking, and beyond.

Vision Skills Needed to Answer Visual Questions

“…We offer our findings as a valuable foundation for demonstrating what types of visual questions need to be included/added in future datasets for the machine learning community. Our work also reinforces the importance of participatory design [33] in creating such crowdsourcing tasks and motivates the importance of changing the status quo, such as by engaging real users with crowd workers to guide them to generate practically relevant data (e.g., extending the work of Venkatagiri et al [55]).…”

The expanding circles of information behavior and human–computer interaction

“…• • Professions outside of computing, such as teaching (Gautam et al, 2017), journalism (Venkatagiri et al, 2019), and medicine (Mentis et al, 2012); • • Interpersonal issues, including the home (Wolf et al, 2019), relationships (Brereton et al, 2015;Moser et al, 2017), privacy (Ackerman and Mainwaring, 2005), and ludic engagement (Gaver et al, 2004); • • Community issues, such as community art (Jacobs et al, 2016), community policing (Israni et al, 2017), and urban design (Mahyar et al, 2016); • • Societal issues, such as development (Dell and Kumar, 2016), politics (Kuznetsov et al, 2011), andsustainability (DiSalvo et al, 2010).…”

“…Professions outside of computing, such as teaching (Gautam et al, 2017), journalism (Venkatagiri et al, 2019), and medicine (Mentis et al, 2012);…”

Mapping Points of Interest Through Street View Imagery and Paid Crowdsourcing

“…A possible direction is to evolve the VCE to support crowd-augmented expert work, a model where experts are simultaneously performing the same (or a superset) of tasks as the crowd, and their own work is shaped and redirected based on the crowd results. This model has been recently applied with success to the task of geolocating images using the concept of shared representations [Venkatagiri et al 2019]. One could also envision a shared representation between a remote crowdworker and a local volunteer engaged through a spatial crowdsourcing application.…”