Towards Computational Assessment of Idea Novelty

Abstract

In crowdsourcing ideation websites, companies can easily collect large amount of ideas. Screening through such volume of ideas is very costly and challenging, necessitating automatic approaches. It would be particularly useful to automatically evaluate idea novelty since companies commonly seek novel ideas. Three computational approaches were tested, based on Latent Semantic Analysis (LSA), Latent Dirichlet Allocation (LDA) and term frequency-inverse document frequency (TF-IDF), respectively. These three approaches were used on three set of ideas and the computed idea novelty was compared with human expert evaluation. TF-IDF based measure correlated better with expert evaluation than the other two measures. However, our results show that these approaches do not match human judgement well enough to replace it.

Let the computer evaluate your idea: evaluation apprehension in human-computer collaboration

“…Computers actively participate in decision-making processes (Jarrahi, 2018) and assess the novelty of ideas (K. Wang et al, 2019). Within the research field of computational creativity, studies and artifacts show how AI-based systems can take over different creative tasks and thus act as active partners in creative processes (Nass et al, 1996).…”

“…In our research, we specifically refer to the process of idea evaluation and whether it would be beneficial to use AI to assess and comment on ideas. AI-based system are already used for idea evaluation in various domains, where the evaluation can be compared to that by a human expert (Maher & Fisher, 2012;Varshney et al, 2019;K. Wang et al, 2019).…”

“…Other approaches for AI-based idea evaluation that exist rely on methods like latent semantic analysis, latent dirichlet allocation and term frequency-inverse document frequency. Such AI-based evaluation methods can be compared to human expert evaluation (K. Wang et al, 2019).…”

I'm creative and deserving! From self‐rated creativity to creative recognition

“…Thus, similar to traditional workplaces, in digital work settings, it might be easier to evaluate and spot the novelty of ideas but not their usefulness. It has been shown that human-technology interaction is enhancing one's novelty in ideas (Shuxin et al, 2017), and novelty overall evokes happiness in people, and it is making things interesting and easier for outsourcers to understand and evaluate (Wang et al, 2019). Such underlying mechanism might not work as effectively when outsourcers need to evaluate the usefulness of gig workers' ideas, because these might need either outsourcers' close expertise in the field to understand the ideas or practical implication of ideas, taking additional time and resources for the outsourcers.…”

Predicting popular contributors in innovation crowds: the case of My Starbucks Ideas

“…Little research has addressed this approach. A growing number of studies shed some light on the semantic features of ideas in crowdsourcing by computational techniques: Schaffhausen and Kowalewski (2016) developed the semantic similarity algorithm to estimate idea similarity in an open innovation platform; Wang et al (2019) used topic modeling to measure idea novelty; Hoornaert et al (2017) applied latent semantic indexing to predict IT product implementation in the Mendeley community; and Rhyn and Blohm (2017) used a machine learning approach to filter the quality of ideas from crowdsourcing. However, at present, research focusing on popular contributor prediction by user-generated content is still largely uninvestigated.…”