We have demonstrated an attentional bias to reminders of the deceased versus a living attachment in grieving. Overlapping neural circuits related to living- and deceased-related attention suggest that the bereaved employ similar processes in attending to the deceased as they do in attending to the living. Deceased-related attentional bias appears to be linked primarily to intrusive thinking about the loss.

End-to-end text-to-speech (TTS) has shown great success on large quantities of paired text plus speech data. However, laborious data collection remains difficult for at least 95% of the languages over the world, which hinders the development of TTS in different languages. In this paper, we aim to build TTS systems for such low-resource (target) languages where only very limited paired data are available. We show such TTS can be effectively constructed by transferring knowledge from a high-resource (source) language. Since the model trained on source language cannot be directly applied to target language due to input space mismatch, we propose a method to learn a mapping between source and target linguistic symbols. Benefiting from this learned mapping, pronunciation information can be preserved throughout the transferring procedure. Preliminary experiments show that we only need around 15 minutes of paired data to obtain a relatively good TTS system. Furthermore, analytic studies demonstrated that the automatically discovered mapping correlate well with the phonetic expertise.

Background Deceased-related thinking is central to grieving and potentially critical to processing of the loss. Self-report measurements might fail to capture important elements of deceased-related thinking and processing. Here, we used a machine learning approach applied to fMRI - known as neural decoding - to develop a measure of ongoing deceased-related processing. Methods 23 subjects grieving the loss of a first-degree relative, spouse or partner within 14 months underwent two fMRI tasks. They first viewed pictures and stories related to the deceased, a living control and a demographic control figure while providing ongoing valence and arousal ratings. Second, they performed a 10-minute Sustained Attention to Response Task (SART) with thought probes every 25–35 seconds to identify deceased, living and self-related thoughts. Results A conjunction analysis, controlling for valence/arousal, identified neural clusters in basal ganglia, orbital prefrontal cortex and insula associated with both types of deceased-related stimuli vs. the two control conditions in the first task. This pattern was applied to fMRI data collected during the SART, and discriminated deceased-related but not living or self-related thoughts, independently of grief-severity and time since loss. Deceased-related thoughts on the SART correlated with self-reported avoidance. The neural model predicted avoidance over and above deceased-related thoughts. Conclusions A neural pattern trained to identify mental representations of the deceased tracked deceased-related thinking during a sustained attention task and also predicted subject-level avoidance. This approach provides a new imaging tool to be used as an index of processing the deceased for future studies of complicated grief.

Background Causal estimation methods are increasingly being used to investigate functional brain networks in fMRI, but there are continuing concerns about the validity of these methods. New Method Multivariate Dynamical Systems (MDS) is a state-space method for estimating dynamic causal interactions in fMRI data. Here we validate MDS using benchmark simulations as well as simulations from a more realistic stochastic neurophysiological model. Finally, we applied MDS to investigate dynamic casual interactions in a fronto-cingulate-parietal control network using Human Connectome Project (HCP) data acquired during performance of a working memory task. Crucially, since the ground truth in experimental data is unknown, we conducted novel stability analysis to determine robust causal interactions within this network. Results MDS accurately recovered dynamic causal interactions with an area under receiver operating characteristic (AUC) above 0.7 for benchmark datasets and AUC above 0.9 for datasets generated using the neurophysiological model. In experimental fMRI data, bootstrap procedures revealed a stable pattern of causal influences from the anterior insula to other nodes of the fronto-cingulate-parietal network. Comparison with Existing Methods MDS is effective in estimating dynamic causal interactions in both the benchmark and neurophysiological model based datasets in terms of AUC, sensitivity and false positive rates. Conclusions Our findings demonstrate that MDS can accurately estimate causal interactions in fMRI data. Neurophysiological models and stability analysis provide a general framework for validating computational methods designed to estimate causal interactions in fMRI. The right anterior insula functions as a causal hub during working memory.

In this paper we propose a Sequential Representation Quantization AutoEncoder (SeqRQ-AE) to learn from primarily unpaired audio data and produce sequences of representations very close to phoneme sequences of speech utterances. This is achieved by proper temporal segmentation to make the representations phonemesynchronized, and proper phonetic clustering to have total number of distinct representations close to the number of phonemes. Mapping between the distinct representations and phonemes is learned from a small amount of annotated paired data. Preliminary experiments on LJSpeech demonstrated the learned representations for vowels have relative locations in latent space in good parallel to that shown in the IPA vowel chart defined by linguistics experts. With less than 20 minutes of annotated speech, our method outperformed existing methods on phoneme recognition and is able to synthesize intelligible speech that beats our baseline model.

Advances in the instrumentation and signal processing for simultaneously acquired electroencephalography and functional magnetic resonance imaging (EEG-fMRI) have enabled new ways to observe the spatiotemporal neural dynamics of the human brain. Central to the utility of EEG-fMRI neuroimaging systems are the methods for fusing the two data streams, with machine learning playing a key role. These methods can be dichotomized into those that are symmetric and asymmetric in terms of how the two modalities inform the fusion. Studies using these methods have shown that fusion yields new insights into brain function that are not possible when each modality is acquired separately. As technology improves and methods for fusion become more sophisticated, the future of EEG-fMRI for noninvasive measurement of brain dynamics includes mesoscale mapping at ultrahigh magnetic resonance fields, targeted perturbation-based neuroimaging, and using deep learning to uncover nonlinear representations that link the electrophysiological and hemodynamic measurements. Expected final online publication date for the Annual Review of Neuroscience, Volume 44 is July 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Network interactions are likely to be instrumental in processes underlying rapid perception and cognition. Specifically, high-level and perceptual regions must interact to balance pre-existing models of the environment with new incoming stimuli. Simultaneous electroencephalography (EEG) and fMRI (EEG/fMRI) enables temporal characterization of brain-network interactions combined with improved anatomical localization of regional activity. In this paper, we use simultaneous EEG/fMRI and multivariate dynamical systems (MDS) analysis to characterize network relationships between constitute brain areas that reflect a subject's choice for a face versus nonface categorization task. Our simultaneous EEG and fMRI analysis on 21 human subjects (12 males, 9 females) identifies early perceptual and late frontal subsystems that are selective to the categorical choice of faces versus nonfaces. We analyze the interactions between these subsystems using an MDS in the space of the BOLD signal. Our main findings show that differences between face-choice and house-choice networks are seen in the network interactions between the early and late subsystems, and that the magnitude of the difference in network interaction positively correlates with the behavioral false-positive rate of face choices. We interpret this to reflect the role of saliency and expectations likely encoded in frontal "late" regions on perceptual processes occurring in "early" perceptual regions. Our choices are affected by our biases. In visual perception and cognition such biases can be commonplace and quite curious-e.g., we see a human face when staring up at a cloud formation or down at a piece of toast at the breakfast table. Here we use multimodal neuroimaging and dynamical systems analysis to measure whole-brain spatiotemporal dynamics while subjects make decisions regarding the type of object they see in rapidly flashed images. We find that the degree of interaction in these networks accounts for a substantial fraction of our bias to see faces. In general, our findings illustrate how the properties of spatiotemporal networks yield insight into the mechanisms of how we form decisions.

Perception and cognition in the brain are naturally characterized as spatiotemporal processes. Decision-making, for example, depends on coordinated patterns of neural activity cascading across the brain, running in time from stimulus to response and in space from primary sensory regions to the frontal lobe. Measuring this cascade is key to developing an understanding of brain function. Here we report on a novel methodology that employs multi-modal imaging for inferring this cascade in humans at unprecedented spatiotemporal resolution. Specifically, we develop an encoding model to link simultaneously measured electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) signals to infer high-resolution spatiotemporal brain dynamics during a perceptual decision. After demonstrating replication of results from the literature, we report previously unobserved sequential reactivation of a substantial fraction of the pre-response network whose magnitude correlates with a proxy for decision confidence. Our encoding model, which temporally tags BOLD activations using time localized EEG variability, identifies a coordinated and spatially distributed neural cascade that is associated with a perceptual decision. In general the methodology illuminates complex brain dynamics that would otherwise be unobservable using fMRI or EEG acquired separately.