Background: Alzheimer’s Disease (AD) can be conceptualized as a continuum: patients progress from normal cognition to mild cognitive impairment (MCI) due to AD, followed by increasing severity of AD dementia. Prior research has measured transition probabilities among later stages of AD, but not for the complete spectrum. Objective: To estimate annual progression rates across the AD continuum and evaluate the impact of a delay in MCI due to AD on the trajectory of AD dementia and clinical outcomes. Methods: Patient-level longitudinal data from the National Alzheimer’s Coordinating Center for n=18,103 patients with multiple visits over the age of 65 were used to estimate annual, age-specific transitional probabilities between normal cognition, MCI due to AD, and AD severity states (defined by Clinical Dementia Rating score). Multivariate models predicted the likelihood of death and institutionalization for each health state, conditional on age and time from the previous evaluation. These probabilities were used to populate a transition matrix describing the likelihood of progressing to a particular disease state or death for any given current state and age. Finally, a health state model was developed to estimate the expected effect of a reduction in the risk of transitioning from normal cognition to MCI due to AD on disease progression rates for a cohort of 65-year-old patients over a 35-year time horizon. Results: Annual transition probabilities to more severe states were 8%, 22%, 25%, 36%, and 16% for normal cognition, MCI due to AD, and mild/moderate/severe AD, respectively, at age 65, and increased as a function of age. Progression rates from normal cognition to MCI due to AD ranged from 4% to 10% annually. Severity of cognitive impairment and age both increased the likelihood of institutionalization and death. For a cohort of 100 patients with normal cognition at age 65, a 20% reduction in the annual progression rate to MCI due to AD avoided 5.7 and 5.6 cases of MCI due to AD and AD, respectively. This reduction led to less time spent in severe AD dementia health states and institutionalized, and increased life expectancy. Conclusion: Transition probabilities from normal cognition through AD severity states are important for understanding patient progression across the AD spectrum. These estimates can be used to evaluate the clinical benefits of reducing progression from normal cognition to MCI due to AD on lifetime health outcomes.

Resting-state functional connectivity (rs-FC) is a promising neuromarker for cognitive decline in aging population, based on its ability to reveal functional differences associated with cognitive impairment across individuals, and because rs-fMRI may be less taxing for participants than task-based fMRI or neuropsychological tests. Here, we employ an approach that uses rs-FC to predict the Alzheimer's Disease Assessment Scale (11 items; ADAS11) scores, which measure overall cognitive functioning, in novel individuals. We applied this technique, connectome-based predictive modeling, to a heterogeneous sample of 59 subjects from the Alzheimer's Disease Neuroimaging Initiative, including normal aging, mild cognitive impairment, and AD subjects. First, we built linear regression models to predict ADAS11 scores from rs-FC measured with Pearson's r correlation. The positive network model tested with leave-one-out cross validation (LOOCV) significantly predicted individual differences in cognitive function from rs-FC. In a second analysis, we considered other functional connectivity features, accordance and discordance, which disentangle the correlation and anticorrelation components of activity timecourses between brain areas. Using partial least square regression and LOOCV, we again built models to successfully predict ADAS11 scores in novel individuals. Our study provides promising evidence that rs-FC can reveal cognitive impairment in an aging population, although more development is needed for clinical application.

Scene content is thought to be processed quickly and efficiently to bias subsequent visual exploration.Does scene content bias spatial attention during task-free visual exploration of natural scenes?If so, is this bias driven by patterns of physical salience or content-driven biases formed through previous encounters with similar scenes? We conducted two eye-tracking experiments to address these questions. Using a novel gaze decoding method, we show that fixation patterns predict scene category during free exploration. Additionally, we isolate salience-driven contributions using computational salience maps and content-driven contributions using gaze-restricted fixation data. We find distinct time courses for salience-driven and content-driven effects. The influence of physical salience peaked initially but quickly fell off at 600 ms past stimulus onset. The influence of content effects started at chance and steadily increased over the 2000 ms after stimulus onset. The combination of these two components significantly explains the time course of gaze allocation during free exploration.

We have begun an exploration of how ubiquitous computing technology can facilitate different forms of audio communication within a family. We are interested in both intra-and inter-home communication. Though much technology exists to support this human-human communication, none of them make effective use of the context of the communication partners. In the Aware Home Research Initiative, we are exploring how to augment a domestic environment with knowledge of the location and activities of its occupants. The Family Intercom project is trying to explore how this context can be used to create a variety of lightweight communication opportunities between collocated and remote family members. It is particularly important that context about the status of the callee be communicated to the caller, so that the appropriate social protocol for continuing a conversation can be performed by the caller. In this paper, we will discuss our initial prototypes to develop a testbed for exploring these context-aware audio communication services.

Eye tracking has long been used to measure overt spatial attention, and computational models of spatial attention reliably predict eye movements to natural images. However, researchers lack techniques to noninvasively access spatial representations in the human brain that guide eye movements. Here, we use functional magnetic resonance imaging (fMRI) to predict eye movement patterns from reconstructed spatial representations evoked by natural scenes. First, we reconstruct fixation maps to directly predict eye movement patterns from fMRI activity. Next, we use a model-based decoding pipeline that aligns fMRI activity to deep convolutional neural network activity to reconstruct spatial priority maps and predict eye movements in a zero-shot fashion. We predict human eye movement patterns from fMRI responses to natural scenes, provide evidence that visual representations of scenes and objects map onto neural representations that predict eye movements, and find a novel three-way link between brain activity, deep neural network models, and behavior.

The extent to which brain functions are localized or distributed is a foundational question in neuroscience. In the human brain, common fMRI methods such as cluster correction, atlas parcellation, and anatomical searchlight are biased by design toward finding localized representations. Here we introduce the functional searchlight approach as an alternative to anatomical searchlight analysis, the most commonly used exploratory multivariate fMRI technique. Functional searchlight removes any anatomical bias by grouping voxels based only on functional similarity and ignoring anatomical proximity. We report evidence that visual and auditory features from deep neural networks and semantic features from a natural language processing model, as well as object representations, are more widely distributed across the brain than previously acknowledged and that functional searchlight can improve model-based similarity and decoding accuracy. This approach provides a new way to evaluate and constrain computational models with brain activity and pushes our understanding of human brain function further along the spectrum from strict modularity toward distributed representation

No abstract

Decoding information from neural responses in visual cortex demonstrates interpolation across repetitions or exemplars. Is it possible to decode novel categories from neural activity without any prior training on activity from those categories? We built zero-shot neural decoders by mapping responses from macaque inferior temporal cortex onto a deep neural network. The resulting models correctly interpreted responses to novel categories, even extrapolating from a single category.Neural decoding approaches typically train machine learning classifiers on 1 responses to a set of stimuli and subsequently test the classifier using either 2 different repetitions of the same training stimuli or responses to different exem-3 plars from the same training categories. These approaches have been extremely 4 successful in a wide variety of domains [1], but show limited generalization. 5 Zero-shot neural decoding, or interpreting neural activity without prior 6 exposure to any similar information [2][3][4][5][6], holds great promise to improve 7 the generalizability of neural information processing models. While standard 8 decoders predict information directly from patterns of neural activity, zero-shot 9 decoders map neural activity to an intermediate representation that constitutes 10 a computational hypothesis for the neural code [2]. The intermediate represen- 11 tation is selected such that it has a known or easily learned relationship to a 12 1/18 wide variety of to-be-predicted outputs. In an impressive recent demonstration 13 of zero-shot decoding, Anumanchipalli and colleagues [6] reconstructed recog-14 nizable human speech from electrophysiological recordings in human motor 15 cortex via a computational model of articulatory movement. Even though the 16 decoding model was only trained to map neural activity to the articulatory 17 model, and not representations of words or semantics, the models could recon-18 struct intelligible human speech. Here, we demonstrate such zero-shot decoding 19 from electrophysiological responses for visual objects. 20 Beyond a feat of engineering, the degree of generalization has important 21 consequences for the conclusions that can be drawn from a model of neural 22 information processing. The greater the generalization, the stronger the evi-23 dence that a model captures generic processing beyond any particular set or 24 class of stimuli. As an example, consider a standard linear decoder trained to 25 distinguish whether responses along the ventral stream were evoked by images 26 of airplanes or chairs. The decoder could interpolate within its training space 27 to label neural responses to new images of airplanes or chairs, but it would 28 not be able to accurately label neural responses to cars or tables. A zero-shot 29 model can capture generic visual information and extrapolate to new categories 30 on which it was not trained. 31 Constructing generic zero-shot decoders for visual objects necessitates a 32 model for visual processing in the primate brain. How well do we understand 33 ...