Information Extraction Meets Crowdsourcing: A Promising Couple

Balke

2015

CIT

Self Cite

In recent years, crowdsourcing has become a powerful tool to bring human intelligence into information processing. This is especially important for Web data which in contrast to well-maintained databases is almost always incomplete and may be distributed over a variety of sources. Crowdsourcing allows to tackle many problems which are not yet attainable using machine-based algorithms alone: in particular, it allows to perform database operators on incomplete data as human workers can be used to provide values during runtime. As this can become costly quickly, elaborate optimization is required. In this paper, we showcase how such optimizations can be performed for the popular skyline operator for preference queries. We present some heuristics-based approaches and compare them to crowdsourcing-based approaches using sophisticated optimization techniques while especially focusing on result correctness.

Section: Crowdsourcingmentioning

confidence: 99%

Section: Crowdsourcingmentioning

confidence: 99%

Crowdsourcing for Query Processing onWeb Data: A Case Study on the Skyline Operator

Maarry

Balke

2015

CIT

Self Cite

“…In previous studies on crowd sourcing it has been shown under certain constraints that missing values in database tuples can be elicited with surprising efficiency and quality as long as the information is generally available [5]. Especially for factual data that can be looked-up on the Web without requiring expert knowledge (e.g., product specifications, telephone numbers, addresses, etc.…”

Section: Crowd-enabled Dbs and Missing Datamentioning

confidence: 99%

“…), the expected data quality is quite high with only a moderate amount of quality assurance (e.g., majority votes). For example, [5] reports that crowd-sourced manual look-ups of movie genres in IMDB.com are correct in ~95% of all cases with costs of $0.03 per tuple (including quality assurance).…”

Section: Crowd-enabled Dbs and Missing Datamentioning

confidence: 99%

Skyline queries in crowd-enabled databases

Proceedings of the 16th International Conference on Extending Database Technology

Maarry

Balke

2013

Self Cite

Skyline queries are a well-established technique for database query personalization and are widely acclaimed for their intuitive query formulation mechanisms. However, when operating on incomplete datasets, skylines queries are severely hampered and often have to resort to highly error-prone heuristics. Unfortunately, incomplete datasets are a frequent phenomenon, especially when datasets are generated automatically using various information extraction or information integration approaches. Here, the recent trend of crowd-enabled databases promises a powerful solution: during query execution, some database operators can be dynamically outsourced to human workers in exchange for monetary compensation, therefore enabling the elicitation of missing values during runtime. Unfortunately, this powerful feature heavily impacts query response times and (monetary) execution costs. In this paper, we present an innovative hybrid approach combining dynamic crowdsourcing with heuristic techniques in order to overcome current limitations. We will show that by assessing the individual risk a tuple poses with respect to the overall result quality, crowdsourcing efforts for eliciting missing values can be narrowly focused on only those tuples that may degenerate the expected quality most strongly. This leads to an algorithm for computing skyline sets on incomplete data with maximum result quality, while optimizing crowd-sourcing costs.

“…Crowdsourcing can often be found in knowledge processing tasks such as data or media classification [8], data acquisition tasks such as data completion [6] or information extraction [16], as well as in providing training data for machine-learning-based approaches [20]. Furthermore, crowdsourcing has proven to be useful to the research community for performing largescale user studies for evaluating new prototype implementations [11], or performing surveys with a large and diverse number of participants for investigating general human behavior or preferences [1].…”

Section: Introductionmentioning

confidence: 99%

Towards Mobile Sensor-Aware Crowdsourcing: Architecture, Opportunities and Challenges

Kunze

Database Systems for Advanced Applications

et al. 2014

Self Cite

Abstract. The recent success of general purpose crowdsourcing platforms like Amazon Mechanical Turk paved the way for a plethora of crowd-enabled applications and workflows. However, the variety of tasks which can be approached via such crowdsourcing platforms is limited by constraints of the web-based interface. In this paper, we propose mobile user interface clients. Switching to mobile clients has the potential to radically change the way crowdsourcing is performed, and allows for a new breed of crowdsourcing tasks. Here, especially the ability to tap into the wealth of precision sensors embedded in modern mobile hardware is a game changer. In this paper, we will discuss opportunities and challenges resulting from such a platform, and discuss a reference architecture.