Indexing dynamic collective constructs using computer-aided text analysis: Construct validity evidence and illustrations featuring team processes.

Abstract

Group for Organizational EffectivenessOrganizational processes have been widely recognized as both multilevel and dynamic, yet traditional methods of measurements limit our ability to model and understand such phenomena. Featuring a popular model of team processes advanced by Marks et al. (2001), we illustrate a method to use individuals' communications as construct valid unobtrusive measures of collective constructs occurring over time. Thus, the purpose of this investigation is to develop computer-aided text analysis (CATA) techniques that can score members' communications into valid team process measures. We apply a deductive content validitybased method to construct CATA dictionaries for Marks et al.'s dimensions. We then demonstrate their convergent validity with subject matter experts' (SMEs) hand-coded team communications and different SMEs' behaviorally anchored rating scales based on video recordings of team interactions, using multitraitmultimethod analyses in two samples. Using a third sample of paramedics performing a high-fidelity mass casualty incident exercise, we further demonstrate the convergent validity of the CATA and SME scorings of communications. We then model the relationships among processes across episodes using all three samples. Next, we test criterion-related validity using a longitudinal dual-discontinuous change growth modeling design featuring the paramedic CATA-scored team processes as related to a dynamic performance criterion. Finally, we integrate behavioral data from wearable sensor badges to illustrate how CATA can be scored at the individual level and then leveraged to model dynamic networks of team interactions. Implications, limitations, directions for the future research, and guidelines for the application of these techniques to other collective constructs are discussed.

Dynamic team composition: A theoretical framework exploring potential and kinetic dynamism in team capabilities.

“…As such, leveraging unobtrusive measurement may play a critical role. While studies utilizing traditional survey methods are able to index several instances of a given process, the use of computer-aided text analysis techniques or advanced analytics can allow for the creation of custom dictionaries which can index communications and interactions at nearly infinitely fine granularity (Cao et al, 2021; Mathieu et al, 2021). There are a number of possible ways to collect these streams of data (e.g., emails, video, activity, proximity trackers).…”

50 Years of Sports Teams in Work Teams Research: Missed Opportunities and New Directions for Studying Team Processes

“…Action processes might be best captured with trained observers coding recordings of teams playing, including conversations during timeouts or on the mound. Most recently, Mathieu et al (2021) introduce computer-aided text analysis (CATA) to score team processes using transcripts of team communications. They find that the robust methodology can serve as a practical way to capture and model multilevel, dynamic team process relationships over time.…”

“…To, Kilduff, Ordonez, and Schweitzer (2018) provide one recent example of the use of data from physiological sensors on individual athletes in a laboratory setting to shed light on a phenomenon they observed using archival NFL data; we found very few other instances of data streams being used to measure dynamic concepts. As emerging technologies such as apps on personal cell phones, wearable devices and sensors, and computer-aided text analysis continue to be used across social science fields (Klonek et al, 2019; Luciano et al, 2018; Mathieu et al, 2021), work groups and teams researchers examining sports team samples will have a wide array of modalities from which to study team processes.…”

Momentum and friction in team interactions

“…To date, with only a few exceptions (Larson et al, 2020;Mathieu et al, 2022) the temporal variations of transition and action processes have not been tested in models of existing research (e.g., Fisher, 2014; nor examined over multiple performance episodes (e.g., ). An exception is a study by Larson and colleagues (2020) who examined a single performance episode and found that student teams performed better when there was a steep increase in transition and action processes towards their deadline.…”

“…Afterwards, teams are expected to shift their engagement to processes that directly contribute to accomplishing the team's goals (i.e., action processes; Marks et al, 2001). Even though Marks et al (2001, p. 361) in transition processes before working on task accomplishment facilitates action processes, while, contrarily, more recent studies found that teams performed better when they did not actively engage in transition processes early on in the performance episode (Larson et al, 2020;Mathieu et al, 2022).…”