Automated Integrative Complexity: Current Challenges and Future Directions

Abstract

Automating integrative complexity is fraught with many challenges. To address these challenges, we discuss the tension between a specificity approach and a more flexible multiple-pass approach, the multifaceted nature of the complexity construct, the gold standard for complexity measurement, the difficulty of human scoring and its consequences for automation, and some ways forward for creating the best complexity measurements. In so doing, we present new data demonstrating (1) initial evidence for the validity of a new automated system for measuring two different forms of complexity (elaborative and dialectical), (2) the danger of constructing measurements in a purely ad hoc fashion that ignores prospective testing, (3) human-to-computer correspondence is in part a function of human-to-human correspondence, (4) human-to-computer correspondence increases systematically as one uses tests with larger units of analysis, and (5) the lack of correspondence of different systems (both human and automated) may occur in part because they were designed for different units of analysis.

Are Conservatives Really More Simple‐Minded than Liberals? The Domain Specificity of Complex Thinking

“…Even if we assume that this prior work on need for closure/structure, openness, and preferences for complex materials represent phenomena on which conservatives are indeed simpler, that would not invalidate the importance of our present findings. As many researchers have pointed out, complexity itself is multifaceted (see, e.g., Conway et al, ; Houck et al, ; Tetlock, Emlen Metz, Scott, & Suedfeld, ). Thus, even if the present results turn out to be limited only to dogmatism and integrative complexity—two aspects of the case being made for conservative simplicity—that would nonetheless suggest for complexity relevant to those types, the general case being made against conservatives in those areas needs revision.…”

“…Someone can think highly complexly about the Iraq War, and yet still think very simply about broccoli. A lot of evidence using many different operations of complex thinking underscores this point (Conway, Schaller, Tweed, & Hallett, ; Houck, Conway, & Gornick, ; Judd & Lusk, ; Liht, Conway, Savage, White, O'Neill, ; Pancer et al, ; Sidanius, ; Suedfeld, ; Tetlock, Peterson, & Lerner, ). For example, the complexity of thinking can be affected by the importance of the content domain (Conway et al, ; Suedfeld, ) by the experience people have with the domain (Conway et al, ; Dasen, ; Suedfeld, ), by the heritability of the domain (Conway, Dodds, Hands Towgood, McClure, & Olson, ), or by the value conflict implied by the domain (e.g., Suedfeld, Bluck, Loewen, & Elkins, ; Tetlock, ).…”

“…As such, the construct is able to capture the underlying mechanisms of the complexity of thought on a broad level that is conceptually independent of the content domain of the passage. It is in part for this domain‐general breadth that integrative complexity is the most widely used scoring system for measuring the complexity of open‐ended statements (e.g., Conway, Conway, Gornick, & Houck, ; Conway & Gornick, ; Conway et al, ; Conway, Suedfeld, & Clements, ; Conway, Suedfeld, & Tetlock, ; Houck et al, ; Suedfeld & Bluck, ; Suedfeld et al, ; Suedfeld & Leighton, ; Suedfeld & Tetlock, ; Tetlock, ).…”

Strategic Communication and the Integrative Complexity‐Ideology Relationship: Meta‐Analytic Findings Reveal Differences Between Public Politicians and Private Citizens in Their Use of Simple Rhetoric

“…While there is a newly developed and validated Automated Integrative Complexity measure (Conway et al, ; Houck et al, ), we opted to only include studies that used the traditional method of hand‐scoring source materials by trained IC coders. This decision was made for two reasons: (1) While evidence suggests Automated Integrative Complexity is frequently used as a proxy of human‐scored complexity, it is nevertheless correlated on average at only .46.…”

“…Why integrative complexity? Integrative complexity has been extensively validated as a measurement of cognitive complexity differences across a number of domains (e.g., Conway & Conway, ; Conway, Conway, Gornick, & Houck, ; Conway, Dodds, Hands Towgood, McClure, & Olson, ; Conway, Suedfeld, & Tetlock, ; Conway et al, ; Conway, Gornick, et al, ; Hale, ; Houck, Conway, & Gornick, ; Suedfeld, , ; Suedfeld & Bluck, ; Suedfeld, Conway, & Eichhorn, ; Suedfeld, Leighton, & Conway, ; Suedfeld & Tetlock, ; Tetlock, , ; Thoemmes & Conway, ).…”