No abstract

In this paper we present the first large-scale scene attribute database. First, we perform crowdsourced human studies to find a taxonomy of 102 discriminative attributes. We discover attributes related to materials, surface properties, lighting, affordances, and spatial layout. Next, we build the "SUN attribute database" on top of the diverse SUN categorical database. We use crowdsourcing to annotate attributes for 14,340 images from 707 scene categories. We perform numerous experiments to study the interplay between scene attributes and scene categories. We train and evaluate attribute classifiers and then study the feasibility of attributes as an intermediate scene representation for scene classification, zero shot learning, automatic image captioning, semantic image search, and parsing natural images. We show that when used as features for these tasks, low dimensional scene attributes can compete with or improve on the state of the art performance. The experiments suggest that scene attributes are an effective low-dimensional feature for capturing high-level context and semantics in scenes.

Hearing loss is a major risk factor for tinnitus, hyperacusis, and central auditory processing disorder. Although recent studies indicate that hearing loss causes neuroinflammation in the auditory pathway, the mechanisms underlying hearing loss–related pathologies are still poorly understood. We examined neuroinflammation in the auditory cortex following noise-induced hearing loss (NIHL) and its role in tinnitus in rodent models. Our results indicate that NIHL is associated with elevated expression of proinflammatory cytokines and microglial activation—two defining features of neuroinflammatory responses—in the primary auditory cortex (AI). Genetic knockout of tumor necrosis factor alpha (TNF-α) or pharmacologically blocking TNF-α expression prevented neuroinflammation and ameliorated the behavioral phenotype associated with tinnitus in mice with NIHL. Conversely, infusion of TNF-α into AI resulted in behavioral signs of tinnitus in both wild-type and TNF-α knockout mice with normal hearing. Pharmacological depletion of microglia also prevented tinnitus in mice with NIHL. At the synaptic level, the frequency of miniature excitatory synaptic currents (mEPSCs) increased and that of miniature inhibitory synaptic currents (mIPSCs) decreased in AI pyramidal neurons in animals with NIHL. This excitatory-to-inhibitory synaptic imbalance was completely prevented by pharmacological blockade of TNF-α expression. These results implicate neuroinflammation as a therapeutic target for treating tinnitus and other hearing loss–related disorders.

In this paper, we discover and annotate visual attributes for the COCO dataset. With the goal of enabling deeper object understanding, we deliver the largest attribute dataset to date. Using our COCO Attributes dataset, a fine-tuned classification system can do more than recognize object categories -for example, rendering multi-label classifications such as "sleeping spotted curled-up cat" instead of simply "cat". To overcome the expense of annotating thousands of COCO object instances with hundreds of attributes, we present an Economic Labeling Algorithm (ELA) which intelligently generates crowd labeling tasks based on correlations between attributes. The ELA offers a substantial reduction in labeling cost while largely maintaining attribute density and variety. Currently, we have collected 3.5 million object-attribute pair annotations describing 180 thousand different objects. We demonstrate that our efficiently labeled training data can be used to produce classifiers of similar discriminative ability as classifiers created using exhaustively labeled ground truth. Finally, we provide baseline performance analysis for object attribute recognition.

We present RNNbow, an interactive tool for visualizing the gradient flow during backpropagation training in recurrent neural networks. RNNbow is a web application that displays the relative gradient contributions from Recurrent Neural Network (RNN) cells in a neighborhood of an element of a sequence. We describe the calculation of backpropagation through time (BPTT) that keeps track of itemized gradients, or gradient contributions from one element of a sequence to previous elements of a sequence. By visualizing the gradient, as opposed to activations, RNNbow offers insight into how the network is learning. We use it to explore the learning of an RNN that is trained to generate code in the C programming language. We show how it uncovers insights into the vanishing gradient as well as the evolution of training as the RNN works its way through a corpus.

Sensory over-responsivity (SOR), extreme sensitivity to or avoidance of sensory stimuli (e.g., scratchy fabrics, loud sounds), is a highly prevalent and impairing feature of neurodevelopmental disorders such as autism spectrum disorders (ASD), anxiety, and ADHD. Previous studies have found overactive brain responses and reduced modulation of thalamocortical connectivity in response to mildly aversive sensory stimulation in ASD. These findings suggest altered thalamic sensory gating which could be associated with an excitatory/inhibitory neurochemical imbalance, but such thalamic neurochemistry has never been examined in relation to SOR. Here we utilized magnetic resonance spectroscopy and resting-state functional magnetic resonance imaging to examine the relationship between thalamic and somatosensory cortex inhibitory (gamma-aminobutyric acid, GABA) and excitatory (glutamate) neurochemicals with the intrinsic functional connectivity of those regions in 35 ASD and 35 typically developing pediatric subjects. Although there were no diagnostic group differences in neurochemical concentrations in either region, within the ASD group, SOR severity correlated negatively with thalamic GABA (r = −0.48, p < 0.05) and positively with somatosensory glutamate (r = 0.68, p < 0.01). Further, in the ASD group, thalamic GABA concentration predicted altered connectivity with regions previously implicated in SOR. These variations in GABA and associated network connectivity in the ASD group highlight the potential role of GABA as a mechanism underlying individual differences in SOR, a major source of phenotypic heterogeneity in ASD. In ASD, abnormalities of the thalamic neurochemical balance could interfere with the thalamic role in integrating, relaying, and inhibiting attention to sensory information. These results have implications for future research and GABA-modulating pharmacologic interventions.

Prior studies have demonstrated that infants and toddlers who later go on to develop autism spectrum disorder (ASD) show atypical functional connectivity as well as altered neural processing of language and other auditory stimuli, but the timeline underlying the emergence of these altered developmental trajectories is still unclear. Here we used resting-state fMRI (rsfMRI) during natural sleep to examine the longitudinal development of functional connectivity in language-related networks from 1.5 to 9 months of age. We found that functional connectivity of networks that underlie the integration of sensory and motor representations, which is crucial for language development, is disrupted in infants at high familial risk (HR) for developing ASD as early as 1.5 months of age. By 9 months of age, HR infants showed hyperconnectivity between auditory and somatosensory regions whereas low risk (LR) infants displayed greater intrahemispheric connectivity between auditory cortex and higher-order temporal regions as well as the hippocampus. Furthermore, while LR infants showed robust changes in functional connectivity during the first year of life with increasing long-range connectivity accompanied by decreasing short-range connectivity over time, HR infants displayed limited developmental changes. Our findings demonstrate that early disruptions in the development of language-related network connectivity may provide an early marker for the later emergence of ASD symptomatology.

A longstanding goal of computer vision is to build a system that can automatically understand a 3D scene from a single image. This requires extracting semantic concepts and 3D information from 2D images which can depict an enormous variety of environments that comprise our visual world. This paper summarizes our recent efforts toward these goals. First, we describe the richly annotated SUN database which is a collection of annotated images spanning 908 different scene categories with object, attribute, and geometric labels for many scenes. This database allows us to systematically study the space of scenes and to establish a benchmark for scene and object recognition. We augment the categorical SUN database with 102 scene attributes for every image and explore attribute recognition. Finally, we present an integrated system to extract the 3D structure of the scene and objects depicted in an image.