Warning: Humans cannot reliably detect speech deepfakes

Abstract

Speech deepfakes are artificial voices generated by machine learning models. Previous literature has highlighted deepfakes as one of the biggest security threats arising from progress in artificial intelligence due to their potential for misuse. However, studies investigating human detection capabilities are limited. We presented genuine and deepfake audio to n = 529 individuals and asked them to identify the deepfakes. We ran our experiments in English and Mandarin to understand if language affects detection performance and decision-making rationale. We found that detection capability is unreliable. Listeners only correctly spotted the deepfakes 73% of the time, and there was no difference in detectability between the two languages. Increasing listener awareness by providing examples of speech deepfakes only improves results slightly. As speech synthesis algorithms improve and become more realistic, we can expect the detection task to become harder. The difficulty of detecting speech deepfakes confirms their potential for misuse and signals that defenses against this threat are needed.

Investigation of Deepfake Voice Detection Using Speech Pause Patterns: Algorithm Development and Validation

“…This permits for an extremely high accuracy in voice clones in a similar domain to the training data but new advancements and subtle changes in these obscure features could soon make these prediction models obsolete. Indeed, when a high-accuracy prediction model was tested on new, out-of-domain voice clones in a recent study, the prediction accuracy was abysmal (AUC is approximately 25%) [ 10 ]. We aimed to evaluate the use of perceptual features in current and future model implementations by testing model performance on a completely new generator.…”

“…For example, the previously mentioned tool that achieved 100% accuracy was trained and tested on a data set of deepfakes generated in 2019, which are of much lower quality than the level of deepfakes available in 2023 [ 8 ]. Furthermore, recent work has shown that out-of-domain voice clone detectors (ie, voice detectors applied outside of the data set in which they were applied) had extremely low performance, obtaining an area under the receiver operator curve (AUC) of 25% [ 10 ]. A more robust detection method might involve searching for the absence of biological features in the cloned voice, rather than the presence of digital features [ 11 ].…”

“…This may result in subtle but detectable differences in the way pauses are present in authentic versus cloned audio. Indeed, when humans were asked to distinguish between audio deepfakes and authentic voices, one of the primary justifications for a fake audio classification was unnatural pauses in the recordings [ 10 ]. Furthermore, when these features were integrated into a classification regime, a moderate accuracy (approximately 85%) was achieved when analyzing deepfakes by perceptual features such as the amplitude of speech and pauses within a recording [ 12 ].…”

Labeling AI-Generated Content: Promises, Perils, and Future Directions

“…An MIT Exploration of Generative AI • From Novel Chemicals to Opera Labeling AI-Generated Content: Promises, Perils, and Future Directions 3 (Farid 2022;Köbis, Doležalová, and Soraperra 2021;Mai et al 2023), and these issues will undoubtedly worsen as technology continues to improve and evolve (Thompson and Hsu 2023;Vynck 2023).…”

Multimodal analysis of disinformation and misinformation

“…Video, of course, relies on both audio and visual channels, and the role of audio itself should not be underestimated. For example, humans cannot reliably detect speech deepfakes [18]. The above-mentioned body of research demonstrates the need to examine dis-/ misinformation from a multimodal perspective.…”