Increased Complexity Has Its Benefits

Abstract

This piece is a response to the Conway, Conway, Gornick, and Houck (2014) description of an automated method for assessing integrative complexity. It builds on the authors' 15 years of experience in automating a variety of textual analysis coding schemes including that measuring conceptual complexity and is intended to address some of the issues raised in the process of automating content analysis categories.KEY WORDS: integrative complexity, conceptual complexity, automated content analysis Conway and his associates have our full and enthusiastic support in their efforts to automate the coding of Integrative Complexity (IC) as automation will increase the quantity and improve the quality, transparency, and replicability of available data. Along the way, they have created their own automated coding infrastructure and encountered a number of obstacles. Fortunately, many of those obstacles can be mitigated, if not completely overcome, by moving from the single-pass coding approach that they are presently using to a multiple-pass approach. Although multiple-pass text-analysis systems cannot solve all the problems Conway, Conway, Gornick, and Houck (2014) have encountered, such a system would make the journey to a satisfactory automated IC both faster and more comfortable. We will tackle a number of the issues that Conway and his colleagues have faced based on more than 15 years of experience automating a variety of text-analysis coding schemes, including the conceptual complexity measure (CC) to which they refer. We will also consider the opportunities that such automation provides for expanding our knowledge about integrative complexity. Examples of Single-and Multiple-Pass SystemsThe IC automated system developed by Conway and his associates (written in Visual Basic underneath an Excel template) appears to be a "single-pass matching" system that relies on matching lists of words or phrases to those presumed to assess various categories of integrative complexity. This general approach, often referred to as a dictionary approach,

Integrative Complexity at Forty: Steps Toward Resolving the Scoring Dilemma

“…Scores can be compared for test‐retest and interscorer reliability, and they can be subjected to standard statistical analyses. Conceptual complexity theory, with its idea of trait complexity as increasing across differentiation and integration, also underlies Conway, Conway, Gornick, and Houck's () and Young and Hermann's () computerized approaches to cognitive complexity scoring, which are described in the current symposium.…”

“…Both Conway et al. 's () and Young and Hermann's () automated approaches are foundations on which closer approximations of human scoring can be built. In fact, the current versions of both appear more promising than previous ones.…”

Integrative Complexity Coding Raises Integratively Complex Issues

“…But, as Conway et al. () show, even the best correlations between human IC codes and automated IC codes are only moderate—and even these values almost certainly understate the divergence (e.g., large correlations between two sets of codes can comfortably coexist with poor agreement between sets, as pointed out by Young & Hermann [] and with large mean differences between scores from one set and from another set).…”

“…Our colleagues Young and Hermann () have suggested that a multipass system can help ameliorate these difficulties, for instance, by coding different parts of speech differently. We also see potential in multipass systems, potential to incorporate more fine‐tuned distinctions between word senses and indications of sentiments.…”

Automated Integrative Complexity: Current Challenges and Future Directions

“…Automating integrative complexity is fraught with many challenges. We would first like to express our sincere gratitude to Suedfeld and Tetlock (), Tetlock, Metz, Scott, and Suedfeld (), and Young and Hermann () for tackling some of these challenges. Their contributions in this symposium spur an important dialogue about the current state of automating complexity and its future possibilities.…”