Sometimes More is Too Much: A Rejoinder to the Commentaries on Greiff et al. (2015)

Abstract

Abstract:In this rejoinder, we respond to two commentaries on the study by Greiff, S.; Stadler, M.; Sonnleitner, P.; Wolff, C.; Martin, R. Sometimes less is more: Comparing the validity of complex problem solving measures. Intelligence 2015, 50, 100-113. The study was the first to address the important comparison between a classical measure of complex problem solving (CPS) and the more recent multiple complex systems (MCS) approach regarding their validity. In the study, we investigated the relations between one classical microworld as the initially developed method (here, the Tailorshop) with three more recently developed multiple complex systems (MCS; here, MicroDYN, Genetics Lab, and MicroFIN) tests. We found that the MCS tests showed higher levels of convergent validity with each other than with the Tailorshop even after reasoning was controlled for, thus empirically distinguishing between the two approaches. The commentary by Kretzschmar and the commentary by Funke, Fischer, and Holt expressed several concerns with how our study was conducted, our data was analyzed, and our results were interpreted. Whereas we acknowledge and agree with some of the more general statements made in these commentaries, we respectfully disagree with others, or we consider them to be at least partially in contrast with the existing literature and the currently available empirical evidence.

Missing the Wood for the Wrong Trees: On the Difficulty of Defining the Complexity of Complex Problem Solving Scenarios

“…Given the diversity captured under this umbrella term, the proposed complexity framework could prove useful, for instance, in reflections on similarities and differences between various approaches to CPS. The current debate around whether less is more [48], or indeed just less [54] or simply not enough [55], or whether more ends up being too much [56] might be one example. A discussion of advantages and disadvantages of “multiple or minimal complex system” approaches (MCS) in comparison to what is now called “classical CPS”, with the system TAILORSHOP as its poster child, could be informed by reflections grounded in the complexity framework.…”

Complex Problem Solving: What It Is and What It Is Not

“…Numerous articles on the subject have been published in recent years, documenting the increasing research activity relating to this field. In the following collection of papers we list only those published in 2010 and later: theoretical papers (Blech and Funke, 2010; Funke, 2010; Knauff and Wolf, 2010; Leutner et al, 2012; Selten et al, 2012; Wüstenberg et al, 2012; Greiff et al, 2013b; Fischer and Neubert, 2015; Schoppek and Fischer, 2015), papers about measurement issues (Danner et al, 2011a; Greiff et al, 2012, 2015a; Alison et al, 2013; Gobert et al, 2015; Greiff and Fischer, 2013; Herde et al, 2016; Stadler et al, 2016), papers about applications (Fischer and Neubert, 2015; Ederer et al, 2016; Tremblay et al, 2017), papers about differential effects (Barth and Funke, 2010; Danner et al, 2011b; Beckmann and Goode, 2014; Greiff and Neubert, 2014; Scherer et al, 2015; Meißner et al, 2016; Wüstenberg et al, 2016), one paper about developmental effects (Frischkorn et al, 2014), one paper with a neuroscience background (Osman, 2012) 1 , papers about cultural differences (Güss and Dörner, 2011; Sonnleitner et al, 2014; Güss et al, 2015), papers about validity issues (Goode and Beckmann, 2010; Greiff et al, 2013c; Schweizer et al, 2013; Mainert et al, 2015; Funke et al, 2017; Greiff et al, 2017, 2015b; Kretzschmar et al, 2016; Kretzschmar, 2017), review papers and meta-analyses (Osman, 2010; Stadler et al, 2015), and finally books (Qudrat-Ullah, 2015; Csapó and Funke, 2017b) and book chapters (Funke, 2012; Hotaling et al, 2015; Funke and Greiff, 2017; Greiff and Funke, 2017; Csapó and Funke, 2017a; Fischer et al, 2017; Molnàr et al, 2017; Tobinski and Fritz, 2017; Viehrig et al, 2017). In addition, a new “Journal of Dynamic Decision Making” (JDDM) has been launched (Fischer et al, 2015, 2016) to give the field an open-access outlet for research and discussion.…”

Overcoming Routine: A 21st Century Skill for a 21st Century Economy

“…There has been a heated controversy on the question how to clearly define and set-up -Complex Problem Solving‖ (CPS) experiments Funke, Fischer, & Holt, 2017;Greiff, Stadler, Sonnleitner, Wolff, & Martin, 2017). There exists agreement that during CPS participants have to overcome barriers stemming from a changing, dynamic problem environment, uncertainty of relevant information and how system states are interconnected.…”