Synthetic versus human voices in audiobooks: The human emotional intimacy effect

Abstract

Human voices narrate most audiobooks, but the fast development of speech synthesis technology has enabled the possibility of using artificial voices. This raises the question of whether the listeners’ cognitive processing is the same when listening to a synthetic or a human voice telling a story. This research aims to compare the listeners’ perception, creation of mental images, narrative engagement, physiological response, and recognition of information when listening to stories conveyed by human and synthetic voices. The results showed that listeners enjoyed stories narrated by a human voice more than a synthetic one. Also, they created more mental images, were more engaged, paid more attention, had a more positive emotional response, and remembered more information. Speech synthesis has experienced considerable progress. However, there are still significant differences versus human voices, so that using them to narrate long stories, such as audiobooks do, is difficult.

I am Alexa, your virtual tutor!: The effects of Amazon Alexa’s text-to-speech voice enthusiasm in a multimedia learning environment

“…Besides visual aesthetics such as anthropomorphism and warm colors (Brom et al, 2018;Wong & Adesope, 2021), social cue qualities like attractiveness, cuteness, and emotional expressions of virtual or human characters in a multimedia learning environment can also influence learners' affective-motivational factors (Domagk, 2010;Domagk et al, 2010;Horovitz & Mayer, 2021;Plass et al, 2020; Schneider et al, 2021). Along this line of reasoning, manipulating voice socio-emotive cues in a multimedia learning environment falls under the Emotional Design model (Beege et al, 2020;Liew et al, 2017;Rodero & Lucas, 2021;Wang et al, 2022). The literature has extolled the benefits of instructors' displayed enthusiasm in promoting learners' affective-motivational behaviors such as willingness to learn, learning enjoyment, and learning performance in classrooms (Keller et al, 2016;Kim & Schallert, 2014;Kunter et al, 2011Kunter et al, , 2013Moè, 2016;Moe et al, 2021).…”

War feels less horrid in a foreign accent: exploring the impact of the foreign accent on emotionality

“…While we regularly encounter synthesized voices in our daily interactions with digital assistants like, for example, Alexa (Amazon) and Siri (Apple Inc.), the synthesized voices used in our study may have somehow affected speech processing compared to real voices. Despite the fact that none of our participants reported noticing that the voices were artificially generated, a recent study reports that audiobook stories narrated by human voices overall are judged as more enjoyable compared with synthesized voices (Rodero and Lucas, 2023). Moreover, participants showed more engagement, paid more attention, remembered more details and created more mental images when listening to human voices.…”

Three minutes of fame: Effects of movie commentary video style on users’ liking and commenting intentions

“…Thus, it is worth using scene in video classi cation. Due to the effective, credible and easily understandable nature of sound, it is better suited to emotional communication, which is particularly important for storytelling (Rodero and Lucas, 2021). Therefore, sound is found to be useful for classifying videos as well.…”

“…Drama movie is a genre of movie with serious presentations or stories with settings or life situations that portray realistic characters in con ict with either themselves, others, or forces of nature (Bacha-Trams et al, 2018). Dramas often, re ecting people's affection, friendship and love, have tragic or at least painful resolutions with profound meaning (Rodero and Lucas, 2021). Viewers are attracted to the plot and characters when watching a drama movie, and reach "a level of consensus" in the sense of being moved (Davis, 2006).…”