Discrimination performance of statistical values used in the concealed information test studies

Abstract

The concealed information test (CIT) estimates whether an examinee knows crime-relevant information on the basis of diŠerences in physiological responses between the crime-relevant and irrelevant information. To examine the diŠerences in physiological responses, this study used the following four statistical values: Lykken's scores, likelihood ratios, eŠect sizes, and p values of randomization tests. A total of 152 participants, 80 of whom actually knew information related to a mock theft, received a CIT about the theft. From these CIT data, the four statistical values were calculated. On the basis of these values, whether each participant knew the theft-related information was ascertained. High correct discrimination ratios were seen for eŠect sizes (92.4) and p values of randomization tests (90.6) compared to Lykken's scores (83.2) and likelihood ratios (74.8). Areas under the receiver operating characteristic curves were also greater for eŠect sizes (0.919) and p values of randomization tests (0.928) than for the other statistical values. These results suggest that eŠect sizes and p values of randomization tests are promising for examining the physiological diŠerences in the CIT.

Broadening the Use of the Concealed Information Test in the Field

“…Considering the above four conditions, Matsuda et al (29) proposed the use of the d value for effect size (30–32) and the p -value of the randomization test (33). Both d and p -values can be simply computed and require no database.…”

“…In this section, we first explain how to compute d values (section Known-Solution CIT) and p -values (section Searching CIT). We then introduce the performances of d and p -values as compared with that of a traditional method (i.e., Lykken scoring) according to Matsuda et al (29) (section Summary of the Threshold). We also compared these performance data with those of recent machine learning methods.…”

“…Matsuda et al (29) assessed the performance of d and p -values using the dataset of Ogawa et al (23). The dataset consists of experimental CIT data from 152 examinees.…”

“…Matsuda et al (29) computed the integrated d and p -values for each CIT, in addition to the integrated Lykken score. Lykken score is a traditional scoring method (4) that assigns 2 to the largest response and 1 to the second-largest response in a block of repetitions, and then summarizes the scores across all blocks.…”

“…Recently, many machine learning methods have been proposed. We applied typical machine learning methods to the same dataset used in Matsuda et al (29) using the Classification Learner App in MATLAB R2018a. This app automatically calculates the performance of various classifiers by protecting against overfitting using cross-validation.…”