ASR Systems as Models of Phonetic Category Perception in Adults

Abstract

We test the potential of standard Automatic Speech Recognition (ASR) systems trained on large corpora of continuous speech as quantitative models of human speech processing. In human adults, speech perception is attuned to efficiently process native speech sounds, at the expense of difficulties in processing non-native sounds. We use ABX-discriminability measures to test whether ASR models can account for the patterns of confusion between speech sounds observed in humans. We show that ASR models reproduce some well-documented effects in non-native phonetic perception. Beyond the immediate results, our methodology opens up the possibility of a more systematic investigation of phonetic category perception in humans.

Comparing unsupervised speech learning directly to human performance in speech perception

“…A few studies have investigated patterns of L2 discrimination in acoustic models, looking at overall accuracy on phonemic contrasts from languages other than the training language. But their conclusions have been based on qualitative summaries of the behaviour of the models, with no human reference data on the same stimuli (Schatz et al, 2017;Schatz & Feldman, 2018).…”

Perceptimatic: A Human Speech Perception Benchmark for Unsupervised Subword Modelling

“…The catch trials consisted of additional, highly distinct ABX stimuli, including several which required participants to distinguish cat from dog for English speakers or caillou from hibou for French speakers. 7 No participant is tested twice on the same phone pair, and the combination of speakers is not predictive of the right answer. 8 If the same sound belongs to different inventories, it is treated as distinct, for a total of 1032 possible phonemes.…”

“…For testing, triplets were counterbalanced into lists of 190 per participant. 7 Each triplet was tested three times, so that most contrasts are tested at least 36 times. Participants respond as to which of the two reference stimuli the probe corresponded to on a six-point scale, ranging from first for sure to second for sure, with two intermediate degrees of certainty for each.…”

“…The finer-grained study in [3] compared Japanese speakers' perceptual boundaries for the (allophonic) [s]/[C] contrast with the behaviour of GMM phoneme classifiers, while [4] compared phonetic adaptation by Dutch listeners to artificially "accented" productions of the [r]/[l] to acoustic models adapted to the experimental stimuli. Several studies also investigate whether ASR and unsupervised speech representation learning accords with established phenomena in human speech perception at a qualitative level [5,6,7,8].…”