Standardizing Procedures for Calculating Rorschach Interrater Reliability: Conceptual and Empirical Foundations

McDowell, Claude J.; Acklin, Marvin W.

doi:10.1207/s15327752jpa6602_9

Cited by 59 publications

(50 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Alternatively, poor inter-rater reliability would prevent consistent interpretation across examiners, thus compromising the utility because of variability in scoring and interpretation. A great deal of research with various Rorschach systems and scores has demonstrated strong inter-rater reliability (Acklin, McDowell, Verschell, & Chan, 2000;McDowell & Acklin, 1996;Exner, 1993;Meyer & Archer, 2001;Viglione, 1999;Viglione & Meyer, 2008;Viglione & Taylor, 2003). Inter-rater reliability for the CS (Exner, 2003) has been particularly wellestablished, with a large-scale meta-analysis of CS inter-rater reliability reporting high inter-rater reliability for the majority of CS scores (specifically, median intraclass correlations for statistically stable scores ranged from .72-.96; Meyer et al, 2002).…”

Section: Inter-rater Reliability R-pasmentioning

confidence: 99%

An Inter-Rater Reliability Study for the Rorschach Performance Assessment System

Viglione

Blume-Marcovici

Miller

et al. 2012

Journal of Personality Assessment

View full text Add to dashboard Cite

In response to research demonstrating limitations in Rorschach validity and reliability, Meyer, Viglione, Mihura, Erard and Erdberg (2011) have developed a new Rorschach System, R-PAS. Based on the available research findings, this system attempts to ground the Rorschach in its evidence base, improve its normative foundation, integrate international findings, reduce examiner variability, and increase utility. As this Rorschach system is new, no reliability studies have yet been produced. The present study sought to establish inter-rater reliability for the new R-PAS. 50 Rorschach records were randomly selected from ongoing research projects using R-Optimized administration. (Wood, 1996) were based on the criticism that CS inter-rater reliability was determined using percent agreement without correcting for chance. However, as noted, inter-rater reliability has been demonstrated with chance-corrected statistics, including ICC, kappa and Iota coefficients, as the most appropriate and precise statistical methods, proving early criticisms to be unfounded. Indeed, inter-rater reliability for the majority of Rorschach scores compare favorably to other published meta-analyses of inter-rater reliability in psychology, psychiatry, and medicine (Meyer, 2004). Given the wide variety of scores, scales, research projects, and systems from which good reliability has been demonstrated, one must conclude that well-trained coders should achieve acceptable, good, and often excellent inter-rater reliability for the great variety of Rorschach scores.As demonstrated by Weiner (2003), the Rorschach can be considered to be a method of generating data relevant to personality and information processing. From this perspective, various scores and scoring methods systematize the data produced during Rorschach administration, thus constituting the Rorschach as a test. As shown by the strong reliability data across different types of systems, scores, countries, and languages, this test has produced consistently strong reliability. The R-PAS includes many variables that were also used in the CS, clarifies and specifies coding instructions, and modifies a few (e.g. Sex content) to be more consistent with their interpretation. Thus, the abovereported research findings suggest that inter-rater reliability for these R-PAS variables should be strong.R-PAS also includes variables not used in the CS (

show abstract

Section: Inter-rater Reliability R-pasmentioning

confidence: 99%

An Inter-Rater Reliability Study for the Rorschach Performance Assessment System

Viglione

Blume-Marcovici

Miller

et al. 2012

Journal of Personality Assessment

View full text Add to dashboard Cite

show abstract

“…Kappa coefficients were calculated to determine the level of agreement between raters. Kappa coefficients were estimated according to the methods of McDowell and Acklin (1996).…”

Section: Ratingsmentioning

confidence: 99%

The mangled butterfly: Rorschach results from 45 violent psychopaths

Franks

Sreenivasan

Spray

et al. 2009

Behavioral Sci & The Law

View full text Add to dashboard Cite

Participants were 45 violent California male prison inmates scoring 30 or more on the Psychopathy Checklist-Revised (PCL-R; Hare, 1991, 2003). Inmates were evaluated using Rorschach and neuropsychological test data. The participants' intellectual functioning was within the low-average range and displayed a lack of flexibility. Rorschach data were not suggestive of chronic narcissism and anger as in other psychopathic samples. This group resembled Exner's normative sample of high Lambda adults. Consistent with previous studies, psychopaths demonstrated poor emotional modulation, diminished reality testing, little interest in people, and virtually no attachment capacity. Most utilized a simplistic, avoidant, and concrete style. This appeared to be consistent with the concrete thinking and fragmentation attributed to the criminal personality. Concrete thinking is based upon literal interpretations of events. Fragmentation is associated with attitudes that are situation specific and self-serving.

show abstract

“…Proponents and critics of the CS have also debated the temporal consistency and the interrater reliability of the CS (Acklin, McDowell, Verschell, & Chan, 2000;Costello, 1999;Ganellen, 2001;Garb et al, 2001;Hibbard, 2003;Hunsley & Bailey, 1999;Lilienfeld et al, 2000;McDowell & Acklin, 1996;Meyer, 1997aMeyer, , 1997bMeyer et al, 2002;Viglione, 1999;Viglione & Hilsenroth, 2001;Weiner, 2001;Wood, Lilienfeld, Nezworski, & Garb, 2001;Wood, Nezworski, & Stejskal, 1996a, 1996bWood, Nezworski, & Stejskal, 1997). In research, the reliability of the CS variables is now mostly calculated by using the intraclass correlation coefficient, ICC, as recommended by among others Acklin et al (2000).…”

Section: The Rorschach Controversymentioning

confidence: 99%

Rorschach Comprehensive Variables and Assessment for Psychodynamic Psychotherapy

Nygren¹

2005

PsycEXTRA Dataset

View full text Add to dashboard Cite

Rationally selected Comprehensive System (CS; Exner, , 1993Exner, , 2003 Rorschach variables and suitability for psychodynamic psychotherapy were investigated in four studies with different designs. The participants comprised three clinical samples (n = 52, 25, and 69) of patients applying for and/or selected for dynamic psychotherapy. With few exceptions the CS variables were the same in the studies, and in all the studies the Rorschach was independent of clinical ratings/assessments and/or selection of patients for treatment. The first study dealt with the relation between 17 rationally selected CS variables and suitability for psychotherapy as represented by ratings of Dynamic Capacity and Ego Strength and by selection of therapy applicants for psychotherapy. The second and third studies concerned CS score differences between groups of patients differing in suitability for dynamic psychotherapy. In the second study, differences for CS variables were predicted for three groups of dynamic psychotherapy patients with mixed diagnoses. In the third study, CS score differences between two groups, differing in suitability but similar in comprising personality disordered patients, were predicted and tested. In the fourth study, correlations were predicted between 18 pre-therapy CS variables and patient post-therapy satisfaction with therapy and also with therapist post-therapy ratings of their patients' suitability for dynamic therapy.For the four studies taken together, positive results were obtained for EA, FC, the D Score, Blends, Zf, F%, MQo, and AG. The results for m, SumT, WSUM6, X-%, S, and FrrF were negative, whereas the results for YFY, FD, COP, and PER were too weak to permit interpretation.The main limitation of the research in this thesis is the lack of psychometric data concerning the clinical ratings whereas the strength is the ecological validity. The results are positive for some of the CS variables selected, mainly concerning control and cognition but also touching upon emotional integration and interpersonal forcefulness. The usefulness of the CS in psychotherapy assessment may be enhanced if only those CS variables that are found valid for the purpose are used and general personality descriptions are avoided.

show abstract

Standardizing Procedures for Calculating Rorschach Interrater Reliability: Conceptual and Empirical Foundations

Cited by 59 publications

References 0 publications

An Inter-Rater Reliability Study for the Rorschach Performance Assessment System

An Inter-Rater Reliability Study for the Rorschach Performance Assessment System

The mangled butterfly: Rorschach results from 45 violent psychopaths

Rorschach Comprehensive Variables and Assessment for Psychodynamic Psychotherapy

Contact Info

Product

Resources

About