Using Games to Create Language Resources: Successes and Limitations of the Approach

Abstract

One of the more novel approaches to collaboratively creating language resources in recent years is to use online games to collect and validate data. The most significant challenges collaborative systems face are how to train users with the necessary expertise and how to encourage participation on a scale required to produce high quality data comparable with data produced by "traditional" experts. In this chapter we provide a brief overview of collaborative creation and the different approaches that have been used to create language resources, before analysing games used for this purpose. We discuss some key issues in using a gaming approach, including task design, player motivation and data quality, and compare the costs of each approach in terms of development, distribution and ongoing administration. In conclusion, we summarise the benefits and limitations of using a gaming approach to resource creation and suggest key considerations for evaluating its utility in different research scenarios.

Creating language resources for under-resourced languages: methodologies, and experiments with Arabic

“…This is related to the technique known as human computation, which aims to create very large-scale resources by, for example, accumulating annotations contributed by many individuals. It is particularly appealing for creating natural language processing resources (see Chamberlain et al 2013;Poesio et al 2013, for example).…”

Mining Tweet Data - Statistic and Semantic Information for Political Tweet Classification

“…There is a lot of documents representing semantic information, e.g., ontologies, lexicons, semantic networks, and so on. We use the lexicalsemantic network called JeuxDeMots (JDM) (Chamberlain et al, 2013). It is based on the serious game principle in order to construct a large lexical-semantic network.…”

Creating Zombilingo , a game with a purpose for dependency syntax annotation

“…5 The quality obtained for this task is satisfactory [5]. However, another part of the game, which concerned the identification of properties, proved to be too complex and gave unsatisfactory results [4].…”

“…resources of reasonable quality at a relatively low cost [4]. However, the complexity of the tasks to be performed in these games is limited, and they basically rely on the general knowledge of the world that the players have.…”