Contrasting MMPI-3 validity scale effectiveness differences across in-person and telehealth administration procedures.

Abstract

Psychological assessment underwent substantive challenges and changes when the COVID-19 pandemic began, and these changes are likely to endure given the rapid growth of telehealth clinical practice and assessment research using virtual procedures. COVID-19-related changes to assessment practices have impacted accordingly how we study overreporting scale functioning, including the modality through which we administer measures. No available research provides direct comparisons of overreporting scale effectiveness within simulation research across in-person and telehealth modalities, despite early support for novel instruments relying on remote procedures within the historic context of the pandemic. We used simulated feigning conditions collected using best telehealth practices to examine if, and how, overreporting scales differed in effectiveness by evaluating mean scores, elevation rates, and classification accuracy statistics, relative to parallel in-person conditions. Results indicate no meaningful differences in scale effectiveness, particularly when exclusion procedures included a posttest questionnaire. Our findings support telehealth assessment practice and the integration of research collected virtually into the traditional, in-person feigning literature. Limitations and future directions are discussed. Public Significance StatementThis study evaluates how research on feigning conducted on the Minnesota Multiphasic Personality Inventory-3 (MMPI-3) using virtual platforms generalizes to in-person studies. Understanding this generalization of knowledge is important because the MMPI-3 was released early during the COVID-19 pandemic, resulting in most early feigning research being conducted virtually. This use of virtual methods was a notable change as prior feigning research on earlier versions of the MMPI was conducted in person.

Clinical assessment in the time of COVID-19: Introduction to the special issue.

“…The work in this special issue has important implications for clinical practice. Studies by Dombrowski et al (2023), Agarwal et al (2023), Kremyar et al (2023), and Perry et al (2023) provide general support for remote testing. Dombrowski et al’s (2023) and Perry et al’s (2023) studies suggest that even some cognitive testing may be possible via remote administration.…”

“…Of course, the MEZURE was designed for remote administration and adaptations of other instruments would be necessary, along with additional research supporting their use with this testing modality. On the other hand, it is likely that the work on the MMPI-3 by Agarwal et al (2023) and Kremyar et al (2023) is fairly generalizable to other similar self-report measures. Proctored testing of self-report is recommended (SPA, 2020), but studies in this issue lend support for the validity of self-report instrument use in teleassessment.…”

“…On the other hand, some MMPI-3 scale scores were higher in protocols administered during COVID-19, as expected, regardless of administration modality. Agarwal et al (2023) focused their attention on the performance of the MMPI-3 overreporting validity scales, examining whether administration modality (in person, remote) moderated the effectiveness of the validity scales to detect feigned psychopathology. Results suggested that the accuracy of detection was invariant to administration modality.…”

Psychometric properties of the Portuguese version of the Personality Assessment Inventory: normative data and reliability

“…So, although it may bring some disadvantages (e.g., increased environmental variability during testing, possible multiple submissions, and bigger drop-out rates), many of these can be easily avoidable (e.g., by asking participants to participate in the study in a particular environment, by asking for personal data, and by using IP address verification). In addition, some studies have shown that selfselected web samples do not differ systematically from samples obtained from other ways (e.g., in-person), even for cognitive and perceptual demanding tasks (Dandurand et al, 2008;Germine et al, 2012;Hilbig, 2016;Corey and Ben-Porath, 2020;Agarwal et al, 2023), although other studies using the PAI suggest that online participants may be somewhat more socially withdrawn than those participating in in-person studies (McCredie and Morey, 2019). Future studies should compare results between the data collected online and in-person, to check for possible differences in the normative data and psychometric properties.…”

“…The results revealed that the normative data and psychometric properties were equivalent regardless of whether the test was administered in-person or remotely, with group differences no greater than two T-score points per scale ( Menton et al, 2022 ). Overall, a growing body of research has supported the use of telehealth alternatives to in-person assessment procedures, with minimal to no loss of scale effectiveness (e.g., Corey and Ben-Porath, 2020 ; Agarwal et al, 2023 ).…”