How Text Mining Algorithms for Crowdsourcing Can Help Us to Identify Today's Pressing Societal Issues

Abstract

Crowdsourcing is increasingly applied in the area of open development with the goal to find solutions for today's pressing societal issues. To solve such wicked problems, manifold solutions need to be found and applied. In contrast to this, most recent research in crowdsourcing focuses on the few winning ideas, ignoring the sheer amount of content created by the community. In this study we address this issue by applying an automated text mining technique to analyze the ideas contributed by the crowd in an initiative tackling plastic pollution. We show that automated text mining approaches reveal numerous possibilities to make use of the so far unused content of IT enabled collaboration projects. We further add insights into how our findings can help researchers and practitioners to accelerate the solution process for today's pressing societal issues.

Exploring Machine-based Idea Landscapes – The Impact of Granularity

“…We generated an adequate clustering of 619 ideas in just a few seconds without any noteworthy costs. This exemplifies how NLP methods like document embeddings can substitute tedious tasks like reading through every idea, meaningfully organize the diverging perspectives shared by the crowd and dramatically reduce human effort [28,32,42]. The efficient and effective representation of crowdsourced ideas facilitates innovation research to learn more about the size and structure of landscapes full of possible solutions, how distant or close solutions are to each other and the properties of individual landscapes at different granularity levels [29].…”

“…For example, Toubia and Netzer [51] used semantic networks to analyze the structure of a large pool of ideas. Other studies relied on topic modeling algorithms to reveal latent themes in crowdsourced idea descriptions to support the search through the solution-related knowledge [6,22,28] and even compared it to human similarity perceptions [53]. All these approaches share the idea of representing texts based on their semantic similarity.…”

“…This has important implications on how to navigate through idea landscapes and control the cognitive demand of searchers when exploring ideas. Furthermore, we also illustrate how modern NLP methods for defining semantic similarities can structure and analyze myriads of solution-related knowledge shared in crowdsourcing contests more efficiently and effectively [28,42].…”

Fighting the wicked problem of plastic pollution and its consequences for developing regions with expert and crowd solutions

“…Research so far mainly explored how to find the single best solutions among all contributions [6,30]. The added value of the multiple ideas generated within a crowdsourcing initiative is often neglected due to the effort necessary to read and analyze the vast amount of ideas [27]. Traditionally, unstructured data is interpreted using qualitative data approaches, such as reading and manual coding, but given the size of data sets obtained any kind of manual analysis is virtually impracticable [33].…”

“…An indefinite number of individuals contributing to problem solving via IT enabled crowdsourcing approaches are especially valuable to induce a holistic understanding and show different perspectives. Within IT enabled crowdsourcing initiatives, practitioners struggle to make use of the data generated, resulting in the consideration of only the winning ideas without utilizing the vast amount of all contributions [6,27,30]. In contrast to the open innovation community which focuses on the saying "diversity trumps ability" [8], traditional R&D and innovation departments heavily rely on direct expert knowledge.…”