Brain-to-brain synchrony in assessing listening effort

Abstract

Evidence from the neuroscience of verbal communication shows that when two people share information (one speaks and the other listens) their brain activities work in synchrony (Silbert et al., 2014). This brain-to-brain synchrony (B-Bsync) is lost when the listener fails to understand the speaker's message. We test the hypothesis that B-Bsync predicts the level of effort involved in auditory processing: strong neural brain-to-brain coupling reflects relatively effortless processing; conversely, the weaker the coupling, the greater the effort, and the worse the processing and comprehension. We propose that B-Bsync affords a more sensitive assessment of listening effort than currently available on the basis of behavioral measures. Using functional near-infrared spectroscopy (fNIRS) and fNIRS-based hyperscanning approach, we analyze patterns of neural activity separately in the speaker and in the listener, and assess statistically the correspondence in their brain activation (the degree of synchronized activation of cortical sites and temporal symmetry). We examine the effects of degraded source (i.e., accented American English in speakers telling several short stories) on B-Bsync, predicting the strongest coupling and the shortest time delay when the accent of the listener matches that of the speaker, followed by regional dialect mismatch and foreign-accent mismatch, respectively.