Audiovisual speech perception was studied in adults with Asperger syndrome (AS), by utilizing the McGurk effect, in which conflicting visual articulation alters the perception of heard speech. The AS group perceived the audiovisual stimuli differently from age, sex and IQ matched controls. When a voice saying /p/ was presented with a face articulating /k/, the controls predominantly heard /k/. Instead, the AS group heard /k/ and /t/ with almost equal frequency, but with large differences between individuals. There were no differences in gaze direction or unisensory perception between the AS and control participants that could have contributed to the audiovisual differences. We suggest an explanation in terms of weak support from the motor system for audiovisual speech perception in AS.

This paper reports a study on the description and categorization of images. The aim of the study was to evaluate existing indexing frameworks in the context of reportage photographs and to find out how the use of this particular image genre influences the results. The effect of different tasks on image description and categorization was also studied. Subjects performed keywording and free description tasks and the elicited terms were classified using the most extensive one of the reviewed frameworks. Differences were found in the terms used in constrained and unconstrained descriptions. Summarizing terms such as abstract concepts, themes, settings and emotions were used more frequently in keywording than in free description. Free descriptions included more terms referring to locations within the images, people and descriptive terms due to the narrative form the subjects used without prompting. The evaluated framework was found to lack some syntactic and semantic classes present in the data and modifications were suggested. According to the results of this study image categorization is based on high-level interpretive concepts, including affective and abstract themes. The results indicate that image genre influences categorization and keywording modifies and truncates natural image description.Keywords: image content, free description, keywording, categorization, image categories, multidimensional scaling, hierarchical cluster analysis IntroductionThe digitalization of image collections has increased the availability of pictorial material for both commercial and research use. There exists a growing body of research into image retrieval and description. The nature of visual information, however, creates some special challenges. The range and type of attributes needed for describing image content is still under debate. Several frameworks have been created, yet their match with natural, unconstrained image descriptions formed by users has not been proved. The issue of attribute granularity is also challenging; on how many semantic levels should image access be provided? The meanings carried by images, the specificity of index terms, as well as the queries made to image collections may be of various levels. A query might request a specific item or an instance of a general category. It might also deal with a topical category of images or specify a particular abstract concept or affective response the image should evoke.The development of content-based image retrieval systems (i.e. systems that use visual image data to perform queries) has been an area of great interest during the last decade, but many challenges still remain. These include defining visual similarity so that it would match the users' mental models of similarity, as well as bridging the semantic gap between the higher-level semantic concepts used by people and the perceptual attributes addressed by the content-based algorithms. Domain and expected users are important in the development of image description and search tools. Systems and description schem...

This paper reports a study on the types of image categories constructed from magazine photographs. A novel sorting procedure was tested with the aim of providing more data on image similarity and possible category overlap. Expert and non-expert participants were compared in their categorizations. The new similarity sorting procedure resulted in an average of 67%-111% increase in similarity data gathered compared to basic free sorting. Categories were constructed on various levels of similarity: image Function, main visual content (People, Objects and Scene), conceptual content (Theme) and descriptors (Story, Affective, Description, Photography and Visual). Most categories were based on the theme and people portrayed in the photograph, and in the case of the expert subjects, image function. Also abstract and syntactic similarity criteria were employed by the subjects. The categories created by each subject showed on average a 35%-53% overlap. Participants also demonstrated a tendency to use multiple similarity criteria simultaneously and to combine terms from different levels in a single category name. These results indicate a need for a multifaceted approach in image categorization.

This paper describes a comparison of categorization criteria for three image genres. Two experiments were conducted, where naïve participants freely sorted stock photographs and abstract/surreal graphics. The results were compared to a previous study on magazine image categorization. The study also aimed to validate and generalize an existing framework for image categorization. Stock photographs were categorized mostly based on the presence of people, and whether they depicted objects or scenes. For abstract images, visual attributes were used the most. The lightness/ darkness of images and their user-evaluated abstractness/representativeness also emerged as important criteria for categorization. We found that image categorization criteria for magazine and stock photographs are fairly similar, while the bases for categorizing abstract images differ more from the former two, most notably in the use of visual sorting criteria. However, according to the results of this study, people tend to use descriptors related to both image content and image production technique and style, as well as to interpret the affective impression of the images in a way that remains constant across image genres. These facets are present in the evaluated categorization framework which was deemed valid for these genres.

The development of visual retrieval methods requires information about user interaction with images, including their description and categorization. This article presents the development of a categorization model for magazine images based on two user studies. In Study 1, we elicited 10 main classes of magazine image categorization criteria through sorting tasks with nonexpert and expert users (N = 30). Multivariate methods, namely, multidimensional scaling and hierarchical clustering, were used to analyze similarity data. Content analysis of category names gave rise to classes that were synthesized into a categorization framework. The framework was evaluated in Study 2 by experts (N = 24) who categorized another set of images consistent with the framework and found it to be useful in the task. Based on the evaluation study the framework was solidified into a model for categorizing magazine imagery. Connections between classes were analyzed both from the original sorting data and from the evaluation study and included into the final model. The model is a practical categorization tool that may be used in workplaces, such as magazine editorial offices. It may also serve to guide the development of computational methods for image understanding, selection of concepts for automatic detection, and approaches to support browsing and exploratory image search.