Shah’s metrics for measuring ideation effectiveness have been used extensively by the engineering design community to quantify the value of designed concepts. Shah measures novelty as the infrequency of an idea relative to a set of ideas. Vargas-Hernandez extended this novelty metric using partial genealogy trees to consider the frequency of ideas that share the same working principle. These genealogy trees capture differences between individual ideas organized by the following levels of abstraction: physical principle, working principle, and embodiment. Shah’s and Vargas-Hernandez’s metrics both require that all ideas be described at the lowest level (embodiment). This approach excludes ideas that are described at higher levels of abstraction. This paper proposes a new novelty metric that extends Vargas Hernandez’s metrics by including the higher levels of the genealogy trees, allowing abstract ideas to be properly evaluated. This paper compares the newly proposed novelty metric to Shah’s and Vargas Hernandez’s metrics using data from a previous study. The study required participants to perform problem-solving tasks in which they submitted a textual list of ideas for how to solve general day-to-day problems. The proposed novelty metric addresses limitations of the previous metrics when applied to the abstract ideas in the data set and meets established metric requirements. The proposed metric also broadens Shah’s metric in a similar manner as Vargas Hernandez but extends it to capture the entire genealogy tree rather than a subset of the tree.
A suitable quality metric is essential to improving ideation effectiveness. Many proposed quality metrics struggle to adequately capture this critical, subjective concept in a reliable and efficient way. This paper shows our development and testing of a quality metric that is meaningful, repeatable, and efficient. This quality metric is a weighted sum of quality dimensions adapted from the literature. The weighting factors for each dimension are adjusted to the specific ideation problem, and we present here a systematic method to quickly determine these weightings by experimental means. We demonstrate repeatability of the quality metric through interrater reliability, we show meaningfulness by comparing with raters’ intuitive interpretation of quality, and we demonstrate efficiency in the rating process. These initial findings show the quality metric has great promise and merits additional testing and refinement in future work.
This paper studies how engineering education might change divergent thinking skills. We hypothesized that people use a higher amount of divergent thinking when a task is unfamiliar. Our previous work developed an online survey to measure divergent ideation in two ways: with one ideation task, equally familiar to both novice and experienced designers, and a second ideation task, familiar only to experienced designers. We sorted ideas from 40 engineering upperclassmen and 40 freshmen into hierarchical categories and scored fluency, flexibility, and originality. The results did not confirm our hypothesis; rather, we found that originality scores were not significantly different between freshman and upperclassmen. Additionally, both groups produced their most-original ideas in the generally-familiar ideation task. Limitations in our methods prevented meaningful conclusions about flexibility, and further study will be necessary to confirm our other conclusions. To better explore factors influencing divergent thinking, we will refine our methods for future work and retest the participants from the freshmen group in a longitudinal study.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.