ObjectiveThe efficacy of spoken language comprehension therapies for persons with aphasia remains equivocal. We investigated the efficacy of a self-led therapy app, ‘Listen-In’, and examined the relation between brain structure and therapy response.MethodsA cross-over randomised repeated measures trial with five testing time points (12-week intervals), conducted at the university or participants' homes, captured baseline (T1), therapy (T2-T4) and maintenance (T5) effects. Participants with chronic poststroke aphasia and spoken language comprehension impairments completed consecutive Listen-In and standard care blocks (both 12 weeks with order randomised). Repeated measures analyses of variance compared change in spoken language comprehension on two co-primary outcomes over therapy versus standard care. Three structural MRI scans (T2-T4) for each participant (subgroup, n=25) were analysed using cross-sectional and longitudinal voxel-based morphometry.ResultsThirty-five participants completed, on average, 85 hours (IQR=70–100) of Listen-In (therapy first, n=18). The first study-specific co-primary outcome (Auditory Comprehension Test (ACT)) showed large and significant improvements for trained spoken words over therapy versus standard care (11%, Cohen’s d=1.12). Gains were largely maintained at 12 and 24 weeks. There were no therapy effects on the second standardised co-primary outcome (Comprehensive Aphasia Test: Spoken Words and Sentences). Change on ACT trained words was associated with volume of pretherapy right hemisphere white matter and post-therapy grey matter tissue density changes in bilateral temporal lobes.ConclusionsIndividuals with chronic aphasia can improve their spoken word comprehension many years after stroke. Results contribute to hemispheric debates implicating the right hemisphere in therapy-driven language recovery. Listen-In will soon be available on GooglePlay.Trial registration numberNCT02540889.

Hand gestures, imagistically related to the content of speech, are ubiquitous in face-to-face communication. Here we investigated people with aphasia's (PWA) processing of speech accompanied by gestures using lesion-symptom mapping. Twenty-nine PWA and 15 matched controls were shown a picture of an object/action and then a video-clip of a speaker producing speech and/or gestures in one of the following combinations: speech-only, gesture-only, congruent speech-gesture, and incongruent speech-gesture. Participants' task was to indicate, in different blocks, whether the picture and the word matched (speech task), or whether the picture and the gesture matched (gesture task). Multivariate lesion analysis with Support Vector Regression Lesion-Symptom Mapping (SVR-LSM) showed that benefit for congruent speechgesture was associated with 1) lesioned voxels in anterior fronto-temporal regions including inferior frontal gyrus (IFG), and sparing of posterior temporal cortex and lateral temporaloccipital regions (pTC/LTO) for the speech task, and 2) conversely, lesions to pTC/LTO and sparing of anterior regions for the gesture task. The two tasks did not share overlapping voxels. Costs from incongruent speech-gesture pairings were associated with lesioned voxels in these same anterior (for the speech task) and posterior (for the gesture task) regions, but crucially, also shared voxels in superior temporal gyri (STG) and middle temporal gyri (MTG), including the anterior temporal lobe. These results suggest that IFG and pTC/LTO contribute to extracting semantic information from speech and gesture, respectively; however, they are not causally involved in integrating information from the two modalities. In contrast, regions in anterior STG/MTG are associated with performance in both tasks and may thus be critical to speechgesture integration. These conclusions are further supported by associations between performance in the experimental tasks and performance in tests assessing lexical-semantic processing and gesture recognition.

Human face-to-face communication is multimodal: it comprises speech as well as visual cues, such as articulatory and limb gestures. In the current study, we assess how iconic gestures and mouth movements influence audiovisual word recognition. We presented video clips of an actress uttering single words accompanied, or not, by more or less informative iconic gestures. For each word we also measured the informativeness of the mouth movements from a separate lipreading task. We manipulated whether gestures were congruent or incongruent with the speech, and whether the words were audible or noise vocoded. The task was to decide whether the speech from the video matched a previously seen picture. We found that congruent iconic gestures aided word recognition, especially in the noise-vocoded condition, and the effect was larger (in terms of reaction times) for more informative gestures. Moreover, more informative mouth movements facilitated performance in challenging listening conditions when the speech was accompanied by gestures (either congruent or incongruent) suggesting an enhancement when both cues are present relative to just one. We also observed (a trend) that more informative mouth movements speeded up word recognition across clarity conditions, but only when the gestures were absent. We conclude that listeners use and dynamically weight the informativeness of gestures and mouth movements available during face-to-face communication.

No abstract

This study investigates, using behavioral and lesion-symptom mapping methods, the impact of visual speech information for word comprehension in aphasia and the neuroanatomic substrates of any benefit.

Hand gestures, imagistically related to the content of speech, are ubiquitous in face-to-face communication. In the first study with people with aphasia (PWA) investigating speech-gesture processing in the brain using lesion-symptom mapping, we investigated the brain regions as well as the lexical-semantic and gesture recognition abilities associated with benefits and costs of multimodal speech-gesture input. Twenty-nine PWA and 16 matched controls were shown a picture of an object/action and then a video-clip of a speaker producing speech-gesture pairs (congruent/incongruent) or only speaking or gesturing. Their task was to indicate, in different blocks, whether the picture and the word matched (Speech task), or whether the picture and the gesture matched (Gesture task). Multivariate lesion analysis with Support Vector Regression Lesion Symptom Mapping (SVR-LSM) showed that benefit for congruent speech-gesture was associated with 1) lesioned voxels in anterior fronto-temporal regions including inferior frontal gyrus (IFG), and sparing of posterior temporal cortex and lateral temporal-occipital regions (pTC/LTO) for the Speech task, and 2) conversely, lesions to pTC/LTO and sparing of anterior regions for the Gesture task. The two tasks did not share overlapping voxels. Costs from incongruent speech-gesture pairings were associated with lesioned voxels in these same anterior (for the Speech task) and posterior (for the Gesture task) regions, but crucially, also shared voxels in superior and middle temporal gyri, including the anterior temporal lobe. These results suggest that IFG and pTC/LTO contribute to extracting semantic information from speech or gesture respectively; however, they are not causally involved in integrating information from the two modalities. In contrast, regions in anterior STG/MTG are associated with performance in both tasks and may thus be critical to speech-gesture integration. These conclusions are further supported by associations between benefits/costs specific to each task and performance in tests assessing lexical-semantic processing and gesture recognition.

Recent advancements in computer vision promise to automate medical image analysis. Rheumatoid arthritis is an autoimmune disease that would profit from computer-based diagnosis, as there are no direct markers known, and doctors have to rely on manual inspection of X-ray images. In this work, we present a multi-task deep learning model that simultaneously learns to localize joints on X-ray images and diagnose two kinds of joint damage: narrowing and erosion. Additionally, we propose a modification of label smoothing, which combines classification and regression cues into a single loss and achieves 5% relative error reduction compared to standard loss functions. Our final model obtained 4th place in joint space narrowing and 5th place in joint erosion in the global RA2 DREAM challenge.