‘Making such bargain’: Transcribe Bentham and the quality and cost-effectiveness of crowdsourced transcription1

Abstract

In recent years, important research on crowdsourcing in the cultural heritage sector has been published, dealing with topics such as the quantity of contributions made by volunteers, the motivations of those who participate in such projects, the design and establishment of crowdsourcing initiatives, and their public engagement value. This article addresses a gap in the literature, and seeks to answer two key questions in relation to crowdsourced transcription: (1) whether volunteers' contributions are of a high enough standard for creating a publicly accessible database, and for use in scholarly research; and (2) if crowdsourced transcription makes economic sense, and if the investment in launching and running such a project can ever pay off. In doing so, this article takes the award-winning crowdsourced transcription initiative, Transcribe Bentham, which began in 2010, as its case study. It examines a large data set, namely, 4,364 checked and approved transcripts submitted by volunteers between 1 October 2012 and 27 June 2014. These data include metrics such as the time taken to check and approve each transcript, and the number of alterations made to the transcript by Transcribe Bentham staff. These data are then used to evaluate the long-term cost-effectiveness of the initiative, and its potential impact upon the ongoing production of The Collected Works of Jeremy Bentham at UCL. Finally, the article proposes more general points about successfully planning humanities crowdsourcing projects, and provides a framework in which both the quality of their outputs and the efficiencies of their cost structures can be evaluated.

Editorial

“…The ongoing proliferation of online content generation and communication technologies creates many opportunities for public sectors, cultural heritage institutions and humanities scholars to engage the crowd in data collection, sharing, analysis, processing and reuse, sensemaking and value co-creation known as crowdsourcing activities in digital humanities. Although the literature on crowdsourcing and collaboration in digital humanities recognizes the opportunities of some topicsfor example, conceptualization of crowdsourcing in GLAMs (Holley, 2010;Oomen and Aroyo, 2011;Ridge, 2016), task characteristics (Carletti et al, 2015;Terras, 2016), user motivation and engagement (Alam and Campbell, 2017;Severson and Sauve, 2019), technological appropriation (Granell and Mart ınez-Hinarejos, 2016;Iranowska, 2019) and quality control and assessment (Causer et al, 2018;McKinley, 2013;Parent and Eskenazi, 2010) it remains relatively silent on the sociocultural acts and contextualization used to advance understandings. In this regard, there is a pressing need to explore and understand theoretical, methodological and practical issues of crowdsourcing and collaboration in digital humanities.…”

The influences of social value orientation and domain knowledge on crowdsourcing manuscript transcription

“…The prior literature on crowdsourcing has highlighted concerns about quality (Causer et al, 2018;Allahbakhsh et al, 2013). Researchers have identified a variety of factors that may lead to poor-quality outcomes in crowdsourcing process, such as badly designed tasks (Gadiraju et al, 2017;Allahbakhsh et al, 2013), unreasonable rewards (Ming and Chen, 2015) and anonymous participants (Kucherbaev, 2016).…”

Correction des données : retour d’expérience sur la plate-forme RECITAL de transcription participative

“…Bien que le recours au crowdsourcing soit désormais usuel dans nombre de projets de transcription (Transcribe Bentham (Martin et al, 2011;Causer et al, 2018), Transcribathon 6 , Testaments de poilus 7 )) en humanités numériques (Warwick et al, 2012;Carletti et al, 2013)…”