Real-time captioning by non-experts with legion scribe

Abstract

Real-time captioning provides people who are deaf or hard of hearing access to speech in settings such as classrooms and live events. The most reliable approach to provide these captions is to recruit an expert stenographer who is able to type at natural speaking rates, but they charge more than $100 USD per hour and must be scheduled in advance. We introduce Legion Scribe (Scribe), a system that allows 3-5 ordinary people who can hear and type to jointly caption speech in real-time. Each person is unable to type at natural speaking rates, and so is asked only to type part of what they hear. Scribe automatically stitches all of the partial captions together to form a complete caption stream. We have shown that the accuracy of Scribe captions approaches that of a professional stenographer, while its latency and cost is dramatically lower.

Crowdsourcing from Scratch: A Pragmatic Experiment in Data Collection by Novice Requesters

“…Systems such as CrowdDB and Qurk use the crowd for a specific purpose: performing queries, sorts, joins, and other database functions (Franklin et al 2011;Marcus et al 2011). Similarly, Legion:Scribe is a specialized system for a particular task: helping non-expert volunteers provide high-quality captions (Lasecki et al 2013). CrowdForge is an example of a more general-purpose system, which allows requesters to follow a partition-map-reduce pattern for breaking down larger needs into microtasks (Kittur et al 2011).…”