A comparison of signal- and event-contingent ambulatory assessment of interpersonal behavior and affect in social situations.

Abstract

Ambulatory assessment (e.g., ecological momentary assessment) is now widely used in psychological research, yet key design decisions remain largely informed by methodological lore as opposed to systematic inquiry. The present study experimentally tested whether signal-(e.g., random prompt) and event-(e.g., complete a survey every time a target event occurs) contingent recording procedures of interpersonal behavior and affect in social situations yield equivalent quality and quantity of data. Participants (N ϭ 286) completed baseline questionnaires, underwent cluster randomization to either a signal-or event-contingent condition, and then completed 1 week of ambulatory assessment, during which they answered questions about their social behavior and affect tied to their social interactions. Conditions were compared on response frequency, means and variances of interpersonal behavior and affect, correlations between interpersonal behavior and affect within-subject, and associations between momentary behavior and affect and baseline variables (e.g., Big Five traits). Results indicated that signaland event-contingent recording techniques provided equivalent data quality, suggesting that researchers can use the 2 methodologies interchangeably to draw conclusions about means, variances, and associations when examining social interactions. However, results also showed that event-contingent recording returned, on average, a higher number of reported social interactions per individual, and this was true for most time periods of the day. Thus, event-contingent recording may hold advantages for studying frequency and timing of social interactions. Public Significance StatementThis study compared two methods for administering surveys to measure social behavior and mood in daily life, using random prompts throughout the day or asking participants to report only when a social interaction occurs. Results suggest that both methods provide similar data quality, although having individuals report when they experience a social interaction may yield a higher quantity of data.

Making the black box transparent: A template and tutorial for (pre-)registration of studies using Experience Sampling Methods (ESM)

“…Many researchers are already making great strides in increasing reproducibility and transparency in the field of ESM research (e.g. Dejonckheere et al, 2018;Heininga et al, 2019, Himmelstein et al, 2019van Roekel et al, 2019;Zhang et al, 2018) and in clinical psychology more broadly (Tackett et al, 2017;Tackett et al, 2019), where much ESM research is conducted. The adoption of open science practices in ESM research is, however, still in its elementary stages.…”

“…Open science practices, including pre-registration of hypotheses and analysis plans on the Open Science Framework prior to data collection and/or analysis (Nosek, Ebersole, DeHaven, & Mellor, 2018), are initiatives that aim to promote scientific transparency, reproducibility, and replicability. Whilst off to a promising start, the implementation of open science approaches to ESM research, including pre-registration, preprinting and sharing of code and/or materials, are only just emerging (e.g., Dejonckheere, Kalokerinos, Bastian, & Kuppens, 2018;Heininga et al, 2019;Himmelstein et al, 2019;van Roekel et al, 2019;Zhang, Smolders, Lakens & Ijsselsteijn, 2018), and there is still some way to go before such practices become widely adopted.…”

“…Encouragingly, recent years have seen a significant elevation in interest and research energy directed towards addressing methodological and statistical issues in ESM (e.g. Eisele et al, 2020;Himmelstein, Woods & Wright, 2019;Houben, Van Den Noortgate, & Kuppens, 2015;Rintala et al, 2018;Schuurman & Hamaker, 2019;Vachon, Viechtbauer, Rintala, & Myin-Germeys, 2019;Wright & Zimmermann, 2019). Some of these advances present ESM researchers with new choices, which all represent key decisions for a study and as such, potential points of variation in methodological and statistical approaches within as well as between studies.…”

Assessing Personality States: What to Consider when Constructing Personality State Measures

“…Secondly, huge differences exist in how state measures have been developed. With few exceptions (Bleidorn, 2009;Himmelstein et al, 2019;Moskowitz, 1994;Newman, Sachs, Stone, & Schwarz, 2019;Ostojic-Aitkens, Brooker, & Miller, 2019;Zimmermann et al, 2019, Study 2), the state measures were not tested and validated in an independent sample, before the data collection of the substantive study. The most common way to developing state measures thus far seems to be to take items or adjectives that were used to assess personality traits and transform them into state measures (e.g., Horstmann, Rauthmann, Sherman, & Ziegler, in revision;Ziegler, Schroeter, Lüdtke, & Roemer, 2018).…”

Evaluating real-time momentary stress and affect in police officers using a smartphone application

“…A systematic review of diet and physical activity in youth reported response latency longer than 15 min in 0 to 40% of studies using electronic devices, compared to 71.7 to 74.1% for paper-and-pencil methods [42]. A study reported an average response latency of 29 min [45], and another reported 19 min [46]. The participants of those studies were youth (e.g., high school or undergraduate students) who probably responded faster to an EMA than our participants.…”