Idea evaluation as a design process: understanding how experts develop ideas and manage fixations

Abstract

Idea evaluation is used to identify and select ideas for development as future innovations. However, approaching idea evaluation as a decision gate can limit the role of the person evaluating ideas, create fixation bias, and underutilise the person’s creative potential. Although studies show that during evaluation experts are able to engage in design activities, it is still not clear how they design and develop ideas. The aim of this study was to understand how experts develop ideas during evaluation. Using the think-aloud technique, we identify different ways in which experts develop ideas. Specifically, we show how experts transform initial idea concepts using iterative steps of elaboration and transformation of different idea components. Then, relying on concept-knowledge theory (C-K theory), we identify six types of reasoning that the experts use during idea evaluation. This helps us to distinguish between three different roles that experts can move between during evaluation: gatekeeper, designer managing fixation, and designer managing defixation. These findings suggest that there is value in viewing idea evaluation as a design process because it allows us to identify and leverage the experts’ knowledge and creativity to a fuller extent.

How to effectively overcome fixation: a systematic review of fixation and defixation studies on the basis of fixation source and problem type

“…Even though no beyond-frame search is enhanced, an advancement in other aspects could still be achieved as long as there is an expansion within the existing frame. This conclusion is supported by a study conducted by Boudier et al (2023), which suggested that experts in the evaluation of design ideas either contribute to the development of a design solution within the frame constructed by the initial idea, or to the engagement in defixation by initiating alternative solutions beyond the existing frame. Accordingly, to ensure the effectiveness of the defixation approach to open-ended problems, an examination of the influence on both fixation and the measures that are related to the aim of the specific study is necessary to identify whether there is an enhancement in beyond-frame searches or within-frame searches.…”

Learning by doing? The relationship between effort, learning effect and product quality during hackathons of novice teams

“…A jury of five industry and research specialists evaluated the product’s quality. The subjective nature of this assessment introduces the possibility of individual biases affecting the scoring and ranking of the products (Stylidis, Wickman, & Söderberg 2020; Boudier et al 2023). Biases can be observed, as professionals in the industry tend to overestimate the significance of attributes they are currently focusing on (Stylidis et al 2020).…”

“…Efforts were made to address this limitation by appointing a diverse jury with individuals from different backgrounds. The industry experts provided explicit feedback to the teams and posed questions after their presentations, following the recommendation of Boudier et al (2023). However, it is essential to acknowledge that biases cannot be eliminated entirely.…”

“…Biases can be observed, as professionals in the industry tend to overestimate the significance of attributes they are currently focusing on (Stylidis et al 2020). Additionally, experts with similar backgrounds might have varying perceptions of ideas (Boudier et al 2023). Efforts were made to address this limitation by appointing a diverse jury with individuals from different backgrounds.…”