Designing a language game for collecting coreference annotation

Abstract

PlayCoref is a concept of an on-line language game designed to acquire a substantial amount of text data with the coreference annotation. We describe in detail various aspects of the game design and discuss features that affect the quality of the annotation.

Data Linking for the Semantic Web

“…This is often strongly problem-specific and not applicable to most problems. • Some games solve the issue using an online multiplayer scheme, where players validate each others' inputs (von Ahn & Dabbish, 2008;Ho, Chang, Lee, Hsu, & Chen, 2009;Hladká et al, 2009;Siorpaes & Hepp, 2008a). However, this often results in the cold start problem due to the lack of players upon initial game deployment who would be willing to play at the same time.…”

“…According to Hunicke et al (2004) players of computer games may be entertained by several game aesthetic factors (Hunicke, LeBlanc, & Zubek, 2004). In games with a purpose these usually include: (1) social experience -interaction with other players (von Ahn & Dabbish, 2008;Tuulos, Scheible, & Nyholm, 2007), (2) selfchallenge -overcoming a player's own previous achievement, joy of reaching the goal (Seneviratne & Izquierdo, 2010;Terry et al, 2009) and (3) competition among players -either in duels or via a ladder system (Hladká et al, 2009;Chamberlain & Poesio, 2009;Ho et al, 2009 ;von Ahn & Dabbish, 2008).…”

Semantics Discovery via Human Computation Games

“…• Some games solve the issue using an online multiplayer scheme, where players validate each others' inputs (Ahn & Dabbish, 2008;Ho, Chang, Lee, Hsu, & Chen, 2009;Hladká et al, 2009;Siorpaes & Hepp, 2008a). However, this often results in the cold start problem due to the lack of players upon initial game deployment who would be willing to play at the same time.…”

“…According to Hunicke et al players of computer games may be entertained by several game aesthetic factors (Hunicke, LeBlanc, & Zubek, 2004). In games with a purpose these usually include: (1) social experience -interaction with other players (Ahn & Dabbish, 2008;Tuulos, Scheible, & Nyholm, 2007), (2) self-challenge -overcoming a player's own previous achievement, joy of reaching the goal (Seneviratne & Izquierdo, 2010 ;Terry et al, 2009) and (3) competition among players -either in duels or via a ladder system (Hladká et al, 2009;Chamberlain & Poesio, 2009;Ho et al, 2009 ;Ahn & Dabbish, 2008).…”

“…(Thaler, Siorpaes, Simperl, & Hofer, 2011) and is maintained in the Insemtives web page ii . We distinguish two major groups of GWAPs based on their purpose: resource annotation -metadata acquisition games for multimedia (Ahn & Dabbish, 2008;Seneviratne & Izquierdo, 2010;Barrington, O'Malley, Turnbull, & Lanckriet, 2009;Ho et al, 2009;Dasdan et al, 2009), textual resources (Chamberlain & Poesio, 2009;Hladká et al, 2009) or web pages (Law, Mityagin, & Chickering, 2009), and domain modeling games, involving tasks like common sense facts collecting (Ahn & Dabbish, 2008;Kuo et al, 2009) and validation (Herdağdelen & Baroni, 2010), term associations acquisition (Vickrey et al, 2008) or ontology alignment (Siorpaes & Hepp, 2008a).…”