Crowdsourcing has the inherent potential to shift tedious work from requesters with limited time (or patience) to an on-demand workforce. However, the time saved by requesters is offset by the time they must spend preparing instructions and refining them to address the ambiguities that typically arise. Hastily written instructions should be welcomed and supported. This paper presents a set of methods that enable workers to cope with unclear or ambiguous instructions and produce high quality results with minimal reliance on the requester. Workers’ intuition about the requester’s needs is leveraged to move onus for answering questions and revising instructions to workers. Our system, WingIt, implements these methods and demonstrates the relative tradeoffs between resolving by questions versus editing instructions directly, and between waiting for immediate response from the requester versus allowing workers to proceed based on their best guess of the requester’s intent.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.