A Procedure for Evaluating Sensitivity to Within-Person Change: Can Mood Measures in Diary Studies Detect Change Reliably?

Abstract

The recent growth in diary and experience sampling research has increased research attention on how people change over time in natural settings. Often however, the measures in these studies were originally developed for studying between-person differences, and their sensitivity to withinperson changes is usually unknown. Using a Generalizability Theory framework, the authors illustrate a procedure for developing reliable measures of change using a version of the Profile of Mood States (POMS; McNair, Lorr, & Droppleman, 1992) shortened for diary studies. Analyzing two data sets, one composed of 35 daily reports from 68 persons experiencing a stressful examination and another composed of daily reports from 164 persons over a typical 28-day period, we demonstrate that three-item measures of anxious mood, depressed mood, anger, fatigue, and vigor have appropriate reliability to detect within-person change processes.Keywords diary studies; daily mood; within; person change; reliability; Generalizability Theory Correspondence concerning this article should be addressed to James A. Cranford, Addiction Research Center and Department of Psychiatry, University of Michigan, 2025 Traverwood Drive, Suite A, Ann Arbor, MI 48105-2194; e-mail: jcranfor@med.umich.edu; Patrick E. Shrout, 6 Washington Place, Mail Room 416b, Department of Psychology, New York University, New York, NY, 10003; e-mail: pat.shrout@nyu.edu; or Niall Bolger, 406 Schermerhorn Hall, 1190 Amsterdam Avenue, Department of Psychology, Columbia University, New York, NY, 10027; e-mail: nb2229@columbia.edu. James A. Cranford, Addiction Research Center, Department of Psychiatry, University of Michigan; Patrick E. Shrout and Masumi Iida, Department of Psychology, New York University; Eshkol Rafaeli, Psychology Department, Barnard College, Columbia University; Tiffany Yip, Department of Psychology, Fordham University; Niall Bolger, Department of Psychology, Columbia University. 2 In principle, it is possible to estimate the between-person reliability coefficient for mood by averaging a randomly selected set of days (R KR ). Because this is not a design that has obvious advantages, the generalizability coefficient is not presented here, but it is R KR available from the authors on request. NIH Public Access Author ManuscriptPers Soc Psychol Bull. Author manuscript; available in PMC 2008 June 6. NIH-PA Author ManuscriptNIH-PA Author Manuscript NIH-PA Author ManuscriptDaily diary designs are increasingly recommended for studying dynamic psychological processes such as emotional states, self-regulation, and appraisals of social situations (Bolger, Davis, & Rafaeli, 2003;. They have the potential to provide high-resolution information about evolving psychological processes, and they minimize retrospection artifacts and biases. Yet, they are not without their own methodological difficulties . Diary studies impose substantial demands on participants, and the burden imposed by protocols that require frequent self-reports may lead to biases in samples of participa...

Social partners and momentary affect in the oldest-old: The presence of others benefits affect depending on who we are and who we are with.

“…For internal consistency, mean Cronbach's α across the 42 within-day assessments was .41 (SD = .10; range = .17-.60) for PA and .67 (SD = .12; range = .30-.82) for NA. Longitudinal reliability of PA and NA were evaluated using four indices based on the Generalizability Theory (Cranford et al, 2006;Shrout & Lane, 2011). R KF = .99 for PA and .99 for NA.…”

Receiving support as a mixed blessing: Evidence for dual effects of support on psychological outcomes.

“…By doing so, we lowered the alpha of the scale, but this is to be expected because Cronbach's alpha underestimates reliability when items tap different aspects of a construct (Raykov, 1998). 4 The between-person reliability is interpreted as the between-person reliability of the average of the measures taken on the same day; the within-person reliability is interpreted as the reliability of change within person throughout the study (see Cranford et al, 2006). 5 Positive mood was also measured using the Profile of Mood States (Lorr & McNair, 1971), but as positive and negative mood have been shown to operate independently of each other we did not condense them into one scale.…”

The perfectionism model of binge eating: Tests of an integrative model.

“…PMOBE ϭ perfectionism model of binge eating; Interpersonal discrep. ϭ Interpersonal discrepancies; Person ϭ variance due to between-persons differences across all days and across all items; Day ϭ variance due to daily differences across all persons and across all items; Item ϭ variance due to responses to scale items across all persons and across all days; Person by day ϭ variance due to between-persons differences on different days across all items; Person by item ϭ variance due to between-persons differences in responses to scale items across all days; Day by item ϭ variance due to daily differences in responses to scale items across all persons; Error ϭ systematic error (i.e., the Person ϫ Day ϫ Item interaction) plus random error (i.e., unknown sources of variance; Cranford et al, 2006). Small (i.e., near zero) negative variances were observed in our estimates of variance components.…”