We report a cross-cultural study designed to investigate crossmodal correspondences between a variety of visual features (11 colors, 15 shapes, and 2 textures) and the five basic taste terms (bitter, salty, sour, sweet, and umami). A total of 452 participants from China, India, Malaysia, and the USA viewed color patches, shapes, and textures online and had to choose the taste term that best matched the image and then rate their confidence in their choice. Across the four groups of participants, the results revealed a number of crossmodal correspondences between certain colors/shapes and bitter, sour, and sweet tastes. Crossmodal correspondences were also documented between the color white and smooth/rough textures on the one hand and the salt taste on the other. Cross-cultural differences were observed in the correspondences between certain colors, shapes, and one of the textures and the taste terms. The taste-patterns shown by the participants from the four countries tested in the present study are quite different from one another, and these differences cannot easily be attributed merely to whether a country is Eastern or Western. These findings therefore highlight the impact of cultural background on crossmodal correspondences. As such, they raise a number of interesting questions regarding the neural mechanisms underlying crossmodal correspondences.

Diffusion tensor imaging was used to compare white matter structure between American monolingual and Spanish-English bilingual adults living in the United States. In the bilingual group, relationships between white matter structure and naturalistic immersive experience in listening to and speaking English were additionally explored. White matter structural differences between groups were found to be bilateral and widespread. In the bilingual group, experience in listening to English was more robustly correlated with decreases in radial and mean diffusivity in anterior white matter regions of the left hemisphere, whereas experience in speaking English was more robustly correlated with increases in fractional anisotropy in more posterior left hemisphere white matter regions. The findings suggest that (a) foreign language immersion induces neuroplasticity in the adult brain, (b) the degree of alteration is proportional to language experience, and (c) the modes of immersive language experience have more robust effects on different brain regions and on different structural features.

The degree to which we perceive real-world objects as similar or dissimilar structures our perception and guides categorization behavior. Here, we investigated the neural representations enabling perceived similarity using behavioral judgments, fMRI and MEG. As different object dimensions co-occur and partly correlate, to understand the relationship between perceived similarity and brain activity it is necessary to assess the unique role of multiple object dimensions. We thus behaviorally assessed perceived object similarity in relation to shape, function, color and background. We then used representational similarity analyses to relate these behavioral judgments to brain activity. We observed a link between each object dimension and representations in visual cortex. These representations emerged rapidly within 200 ms of stimulus onset. Assessing the unique role of each object dimension revealed partly overlapping and distributed representations: while color-related representations distinctly preceded shape-related representations both in the processing hierarchy of the ventral visual pathway and in time, several dimensions were linked to high-level ventral visual cortex. Further analysis singled out the shape dimension as neither fully accounted for by supra-category membership, nor a deep neural network trained on object categorization. Together our results comprehensively characterize the relationship between perceived similarity of key object dimensions and neural activity.

Colors and odors are associated; for instance, people typically match the smell of strawberries to the color pink or red. These associations are forms of crossmodal correspondences. Recently, there has been discussion about the extent to which these correspondences arise for structural reasons (i.e., an inherent mapping between color and odor), statistical reasons (i.e., covariance in experience), and/or semantically-mediated reasons (i.e., stemming from language). The present study probed this question by testing color-odor correspondences in 6 different cultural groups (Dutch, Netherlands-residing-Chinese, German, Malay, Malaysian-Chinese, and US residents), using the same set of 14 odors and asking participants to make congruent and incongruent color choices for each odor. We found consistent patterns in color choices for each odor within each culture, showing that participants were making non-random color-odor matches. We used representational dissimilarity analysis to probe for variations in the patterns of color-odor associations across cultures; we found that US and German participants had the most similar patterns of associations, followed by German and Malay participants. The largest group differences were between Malay and Netherlands-resident Chinese participants and between Dutch and Malaysian-Chinese participants. We conclude that culture plays a role in color-odor crossmodal associations, which likely arise, at least in part, through experience.

Analyzing functional magnetic resonance imaging (fMRI) pattern similarity is becoming increasingly popular because it allows one to relate distributed patterns of voxel activity to continuous perceptual and cognitive states of the human brain. Here we show that fMRI pattern similarity estimates are severely affected by temporal pattern drifts in fMRI data -even after voxelwise detrending. For this particular dataset, the drift effect obscures orientation information as measured by fMRI pattern dissimilarities. We demonstrate that orientation information can be recovered using three different methods: 1. Regressing out the drift component through linear modeling; 2. Computing representational distances between conditions measured in independent imaging runs; 3. Crossvalidation of pattern distance estimates. One possible source of temporal pattern drift could be random walk like fluctuations -physiological or scanner related -occurring within single voxel timecourses. This explanation is consistent with voxel-wise detrending not alleviating pattern drift effects. In addition, this would explain why crossvalidated pattern distances are robust to temporal drift because a random walk process is expected to give rise to non-replicable drift directions. Given these findings, we recommend that future fMRI studies take pattern drift into account when analyzing pattern similarity as this can greatly enhance the sensitivity to experimental effects of interest.

No abstract

Research into representation learning models of lexical semantics usually utilizes some form of intrinsic evaluation to ensure that the learned representations reflect human semantic judgments. Lexical semantic similarity estimation is a widely used evaluation method, but efforts have typically focused on pairwise judgments of words in isolation, or are limited to specific contexts and lexical stimuli. There are limitations with these approaches that either do not provide any context for judgments, and thereby ignore ambiguity, or provide very specific sentential contexts that cannot then be used to generate a larger lexical resource. Furthermore, similarity between more than two items is not considered. We provide a full description and analysis of our recently proposed methodology for large-scale data set construction that produces a semantic classification of a large sample of verbs in the first phase, as well as multiway similarity judgments made within the resultant semantic classes in the second phase. The methodology uses a spatial multi-arrangement approach proposed in the field of cognitive neuroscience for capturing multi-way similarity judgments of visual stimuli. We have adapted this method to handle polysemous linguistic stimuli and much larger samples than previous work.We specifically target verbs, but the method can equally be applied to other parts of speech. We perform cluster analysis on the data from the first phase and demonstrate how this might be useful in the construction of a comprehensive verb resource. We also analyze the semantic information captured by the second phase and discuss the potential of the spatially induced similarity judgments to better reflect human notions of word similarity.We demonstrate how the resultant data set can be used for fine-grained analyses and evaluation of representation learning models on the intrinsic tasks of semantic clustering and semantic similarity. In particular, we find that stronger static word embedding methods still outperform lexical representations emerging from more recent pre-training methods, both on word-level similarity and clustering. Moreover, thanks to the data set’s vast coverage, we are able to compare the benefits of specializing vector representations for a particular type of external knowledge by evaluating FrameNet- and VerbNet-retrofitted models on specific semantic domains such as “Heat” or “Motion.”

Pre-publication peer review of scientific literature in its present state suffers from a lack of evaluation validity and transparency to the community. Inspired by social networks, we propose a framework for the open exchange of post-publication evaluation to complement the current system. We first formulate a number of necessary conditions that should be met by any design dedicated to perform open scientific evaluation. To introduce our framework, we provide a basic data standard and communication protocol. We argue for the superiority of a provider-independent framework, over a few isolated implementations, which allows the collection and analysis of open evaluation content across a wide range of diverse providers like scientific journals, research institutions, social networks, publishers websites, and more. Furthermore, we describe how its technical implementation can be achieved by using existing web standards and technology. Finally, we illustrate this with a set of examples and discuss further potential.Keywords: open evaluation, peer review, social networking, standard INTRODUCTIONThe success of scientific ideas critically depends on their successful publication. An unpublished idea, innovative, and promising as it might be, remains just that; only after publication it becomes a legitimate part of the scientific consciousness. A central gatekeeper function between the multiplicity of ideas and their manifestation as scientific publications is assigned to formal reviews governed by scientific journals. The current publishing system hinges on voluntary pre-publication peer review, with reviewers selected by the editorial staff. Peer review is undeniably a vital means of research evaluation for it is based on mutual exchange of expertise. Its role in the current system, however, has been the subject of concern with regard to accuracy, fairness, efficiency, and the ability to assess the long-term impact of a publication for the scientific community (Casati et al., 2010). For instance, studies suggest that peer review does not significantly improve manuscript quality (Goodman et al., 1994;Godlee et al., 1998) and that it is susceptible to biases to affiliation (Peters and Ceci, 1982) and gender (Wenneras and Wold, 1997). These concerns seem to be partly caused by the fact that the reviewer selection only includes a small sample from all peers potentially available. Aggravating this situation, no common agreements exist to provide reviewers with uniform guidelines, let alone binding rules, and no established standards by which those rules can be designed-peer review is in fact largely conducted at the discretion of the reviewers themselves. Given that reviewers vary considerably with respect to assessment and strictness, manuscript evaluation in the present model is highly dependent on reviewer selection. The lack of validity is further compounded by review and reviewer confidentiality, rendering them elusive to follow-up inspection.Having been published, a scientific paper is exposed to interested scholars an...