Fixing the stimulus-as-a-fixed-effect fallacy in forensically valid face-composite research.

Abstract

Face composites from eyewitnesses' memories are a valuable resource in tackling crime. Many studies have focused on identifying the best system to produce a nameable composite. In this article, it is described that how many of these studies do not provide reliable conclusions because they fail to treat the faces constructed as being a random factor and so make the stimulus-as-a-fixed-effect fallacy. Simulations are reported in which the statistical methodologies typically employed in these studies are performed on random data generated by a null effect. The first simulation shows that the typical analysis of variance (ANOVA) analysis in this field produces a significant effect (i.e., Type 1 error) 20% of the time. A further simulation shows that using generalized estimating equations (GEE) analysis (recently employed in this type of research) does not resolve the problem. Recommendations are made for the analysis of face-composite experiments to best evaluate and hence improve the quality of the face composites made by eyewitnesses.

How to run linear mixed effects analysis for pairwise comparisons? A tutorial and a proposal for the calculation of standardized effect sizes

“…We did not expect any differences between them and did not want to extend our conclusions to other stimuli. However, for many research questions in psychology we want to make claims that are true not only for all people of the population tested but also for all stimuli of the kinds we presented (Brysbaert & Stevens, 2018;Judd et al, 2012;Lewis, 2023). The stimuli we presented were only a random sample of the stimuli we could have presented and for which we want to make conclusions.…”

“…The first analysis is typically called F1 analysis (over participants); the second F2 analysis (over stimuli). A better alternative, however, is to run a LME analysis with both participants and stimuli as random variables (Brysbaert & Stevens, 2018;Judd et al, 2012;Lewis, 2023).…”

Seeing emotions in the eyes: A validated test to study individual differences in the perception of basic emotions

“…This was a weakness in Swain et al (2022), who used only four faces from an existing database: two from male actors and two from female actors, showing different emotions. A low number of stimuli carries a significant risk that the findings are limited to the stimulus set used and do not generalize to other stimuli (Lewis, 2023;Westfall et al, 2014). Looking at the publications listed above, we see that Swain et al (2022) were not an exception: Ten out of the 16 papers used 10 or fewer face stimuli.…”