Image semantics in the description and categorization of journalistic photographs

Abstract

This paper reports a study on the description and categorization of images. The aim of the study was to evaluate existing indexing frameworks in the context of reportage photographs and to find out how the use of this particular image genre influences the results. The effect of different tasks on image description and categorization was also studied. Subjects performed keywording and free description tasks and the elicited terms were classified using the most extensive one of the reviewed frameworks. Differences were found in the terms used in constrained and unconstrained descriptions. Summarizing terms such as abstract concepts, themes, settings and emotions were used more frequently in keywording than in free description. Free descriptions included more terms referring to locations within the images, people and descriptive terms due to the narrative form the subjects used without prompting. The evaluated framework was found to lack some syntactic and semantic classes present in the data and modifications were suggested. According to the results of this study image categorization is based on high-level interpretive concepts, including affective and abstract themes. The results indicate that image genre influences categorization and keywording modifies and truncates natural image description.Keywords: image content, free description, keywording, categorization, image categories, multidimensional scaling, hierarchical cluster analysis IntroductionThe digitalization of image collections has increased the availability of pictorial material for both commercial and research use. There exists a growing body of research into image retrieval and description. The nature of visual information, however, creates some special challenges. The range and type of attributes needed for describing image content is still under debate. Several frameworks have been created, yet their match with natural, unconstrained image descriptions formed by users has not been proved. The issue of attribute granularity is also challenging; on how many semantic levels should image access be provided? The meanings carried by images, the specificity of index terms, as well as the queries made to image collections may be of various levels. A query might request a specific item or an instance of a general category. It might also deal with a topical category of images or specify a particular abstract concept or affective response the image should evoke.The development of content-based image retrieval systems (i.e. systems that use visual image data to perform queries) has been an area of great interest during the last decade, but many challenges still remain. These include defining visual similarity so that it would match the users' mental models of similarity, as well as bridging the semantic gap between the higher-level semantic concepts used by people and the perceptual attributes addressed by the content-based algorithms. Domain and expected users are important in the development of image description and search tools. Systems and description schem...

A comparative study of Flickr tags and index terms in a general image collection

“…In addition, both Flickr users and indexers were focusing on the description of perceptual attributes and visual characteristics of the images more than interpretive and reactive attributes. These findings are in line with previous research that showed naïve image users using more terms that describe perceptual attributes when viewing individual images (e.g., Jörgensen, 1995Jörgensen, , 1998Laine-Hernandez & Westman, 2006) and using more terms that describe interpretive attributes when labeling groups of images (e.g., Rorissa, 2008;Rorissa & Hastings, 2004;Rorissa & Iyer, 2008).…”

The effect of page context on magazine image categorization

“…Cluster analysis was done with the average-linkage method used also by Rorissa and Hastings (2004). This produced better results for the current data sets than the complete-linkage method used by Laine-Hernandez and Westman (2006), Lohse et al (1990), Teeselink et al (2000, and Vailaya et al (1998).The quality of the solution was evaluated by calculating the cophenetic correlation coefficient which should be close to 1 for a high-quality solution. The coefficient value of the clustering solution was 0.92 for the context group and 0.81 for the no context group.…”

Development and evaluation of a multifaceted magazine image categorization model

“…Multivariate analysis methods seek to represent the similarity structures in the image set. Methods used in image categorization research include nonmetric multidimensional scaling (MDS; used, for example, by Laine-Hernandez & Westman, 2006, Lohse, Rueter, Biolsi, & Walker, 1990, and Rogowitz et al, 1998 and hierarchical cluster analysis (conducted, for example, by Laine-Hernandez & Westman, 2006, Lohse, Biolsi, Walker, & Rueter, 1994, Vailaya et al, 1998, and Teeselink et al, 2000. MDS seeks to find a lowdimensional coordinate system to represent the objects using a similarity matrix as data.…”

“…Markkula and Sormunen (2000) conducted a qualitative study about indexing practices of archivists in a newspaper finding that indexing focused on the concrete content of the photograph. Categorization of journalistic images may also be based on higher level interpretive concepts, including affective and abstract themes (Laine-Hernandez & Westman, 2006). No common frameworks exist for the categorization of journalistic imagery.…”