Variability in the perception of race through voices and faces

Abstract

Although perception of race is known to influence speech intelligibility, speech has often been studied without careful attention to this variability. It has been common to study language in an audio-only modality, which fails to capture the multimodal natures of language and person perception. We have developed a set of audiovisual study materials consisting of participants producing individual sentences selected from two different sets of standard materials. The corpus we have created is intentionally diverse with respect to racial self-report. However, there is extraordinary diversity in how persons who share a racial identification may present. The complex and variable relationship between physical features and the construction of social status demands that we reject reliance on self-reports when characterizing the diversity of our sample. In two experiments, participants viewed faces and listened to voices of people who contributed to the audiovisual corpus. The results allow us to explicitly characterize how the participants in the corpus may be racialized by observers, independent of their self-report. We argue that examining cognitive processes, which give rise to racialized and gendered perceptions, is foundational to advancing our understanding of speech perception in realistic environments with high degrees of variability in personal appearance and linguistic behavior.