Diagnostic reliability of MMPI-2 computer-based test interpretations.

Abstract

Reflecting the common use of the MMPI-2 to provide diagnostic considerations, computer-based test interpretations (CBTIs) also typically offer diagnostic suggestions. However, these diagnostic suggestions can sometimes be shown to vary widely across different CBTI programs even for identical MMPI-2 profiles. The present study evaluated the diagnostic reliability of 6 commercially available CBTIs using a 20-item Q-sort task developed for this study. Four raters each sorted diagnostic classifications based on these 6 CBTI reports for 20 MMPI-2 profiles. Two questions were addressed. First, do users of CBTIs understand the diagnostic information contained within the reports similarly? Overall, diagnostic sorts of the CBTIs showed moderate inter-interpreter diagnostic reliability (mean r = .56), with sorts for the 1/2/3 profile showing the highest inter-interpreter diagnostic reliability (mean r = .67). Second, do different CBTIs programs vary with respect to diagnostic suggestions? It was found that diagnostic sorts of the CBTIs had a mean inter-CBTI diagnostic reliability of r = .56, indicating moderate but not strong agreement across CBTIs in terms of diagnostic suggestions. The strongest inter-CBTI diagnostic agreement was found for sorts of the 1/2/3 profile CBTIs (mean r = .71). Limitations and future directions are discussed.

Interpretive Reliability of Six Computer-Based Test Interpretation Programs for the Minnesota Multiphasic Personality Inventory–2

“…When examining the reliability of q-sorts based on blind MMPI-2 interpretations with clinical psychology doctoral student raters, Robinson (2004) found acceptable sort–resort and pairwise reliabilities (.67 and .38 overall, respectively) with a q-sorting system that served as a precursor to the Midwestern Q-sort. Moreover, reliabilities have been shown to increase following the aggregation of ratings (Marshall, 2001; Pant et al, 2014), as was done in this study. The issue of generalizability would be of greater concern had the raters been asked to interpret MMPI-2 profiles as opposed to providing rankings of the extent to which provided CBTI statements are characteristic of an individual.…”

“…The q-sort method has been used in many studies examining the reliability of personality assessment (e.g., Crumpton, 1975; Deskovitz, 2003; Harrington, 1999; Little & Schneidman, 1959; Moos, 1962; Pant et al, 2014). This procedure requires judges to sort a set of items or statements into ordered categories, ranging from extremely characteristic or salient to extremely uncharacteristic (Ozer, 1993).…”

“…When available, interinterpreter reliabilities have typically been more satisfactory than validity coefficients. In a study of diagnostic reliability, Pant, McCabe, Deskovitz, Weed, and Williams (2014) compared ratings of the psychiatric diagnoses most likely to be appropriate for examinees based on CBTI reports from the same six programs under consideration within the present study. These authors found moderate interinterpreter diagnostic reliability overall (mean r = .56) with a range from .46 (2/4/7 profile) to .67 (1/2/3 profile).…”

Fifty psychological and psychiatric terms to avoid: a list of inaccurate, misleading, misused, ambiguous, and logically confused words and phrases

“…So-called “objective” personality tests are also often subjective with respect to interpretation ( Rogers, 2003 ). For example, even different computerized MMPI-2 interpretive programs display only moderate levels of inter-rater agreement regarding proposed diagnoses ( Pant et al, 2014 ). Not surprisingly, clinicians routinely disagree in their interpretations of profiles on the MMPI-2 and other “objective” tests ( Garb, 1998 ).…”