Validity of content‐based techniques for credibility assessment—How telling is an extended meta‐analysis taking research bias into account?

Abstract

Content-based techniques for credibility assessment (Criteria-Based Content Analysis [CBCA], Reality Monitoring [RM]) have been shown to distinguish between experience-based and fabricated statements in previous meta-analyses. New simulations raised the question whether these results are reliable revealing that using metaanalytic methods on biased datasets lead to false-positive rates of up to 100%. By assessing the performance of and applying different bias-correcting meta-analytic methods on a set of 71 studies we aimed for more precise effect size estimates. According to the sole bias-correcting meta-analytic method that performed well under a priori specified boundary conditions, CBCA and RM distinguished between experience-based and fabricated statements. However, great heterogeneity limited precise point estimation (i.e., moderate to large effects). In contrast, Scientific Content Analysis (SCAN)-another content-based technique tested-failed to discriminate between truth and lies. It is discussed how the gap between research on and forensic application of content-based credibility assessment may be narrowed.

“What works?” Systematic reviews and meta‐analyses of the investigative interviewing research literature

“…Four papers in the special issue explore various aspects of interviewing to detect deception, including a meta-analysis of the Verifiability Approach (Verschuere et al, 2021), an assessment of how cognitive approaches to interviewing can facilitate judgments of deception (Mac Giolla & Luke, 2021), an extended analysis of contentbased techniques for credibility assessment (Oberlader et al, 2021), and a meta-analysis of experimental and field studies evaluating the comparison question polygraph test (Honts et al, 2021). First,…”

“…In some instances, replication of findings was encouraged to extend beyond well-defined research groups (Luke, 2021;Verschuere et al, 2021). Scholars also called for improved method variance that might expand upon the prevailing paradigms and allow for greater generalizability (Luke, 2021;Oleszkiewicz & Watson, 2021), as well as the standardization of coding or measurement procedures and the development of meaningful (practical) measures that more readily translate to investigative practice (Luke, 2021;Oberlader et al, 2021).…”

“…A second set of recommendations related to the need to explore contextual moderators important to ecological validity (Gabbert et al, 2021;Horry et al, 2021;Mac Giolla & Luke, 2021;Oberlader et al, 2021;Otgaar et al, 2021;Verschuere et al, 2021). For example, many of the current experimental paradigms lack elements that are central to the experience of witnesses, victims, sources, or suspects, including the degree of perceived stress or arousal, the variety of motivations associated with providing information, or the complexity of the situations that are so often the purpose of an interview.…”

“…Finally, anyone who has conducted a systematic review or metaanalysis is familiar with the challenges of accessing relevant details associated with a study's methodology or analyses. Scholars in the current issue have similarly called for improvements to the clear and transparent reporting of such information, recommending that materials and data be made available on repositories such as Open Science Framework, and that authors consider limiting potential researcher bias by pre-registering their studies (Horry et al, 2021;Oberlader et al, 2021). Finally, researchers should consider facilitating access to unpublished studies by posting them to repositories such as PsyArXiv.…”

“…Oberlader et al (2021) examined the Criteria Based Content Analysis and Reality Monitoring approaches to content-based credibility…”