This article presents the design and functionalities of interactive software for multimodal analysis currently being developed in the Multimodal Analysis Lab, Interactive Digital Media Institute (IDMI) at the National University of Singapore. The software is being used for the annotation, analysis, search and retrieval of semantic patterns in unified but complex semiotic actsfor example, the interaction of gesture, gaze, intonation, camera angle, and music in a film. In addition to providing a digital platform for multimodal analysis, the software provides the site for further development of multimodal theory as the analytical techniques and tools produce insights into the nature of the multimodal phenomena. The approach is located within the digital humanities paradigm that promotes the use of computer techniques and technologies for humanities, arts, and social science research.
The paper discusses the challenges faced by researchers in developing effective digital interfaces for analyzing the meaning-making processes of multimodal phenomena. The authors propose a social semiotic approach as the underlying theoretical foundation, because interactive digital technology is the embodiment of multimodal social semiotic communication. The paper outlines the complex issues with which researchers are confronted in designing digital interface frameworks for modeling, analyzing, and retrieving meaning from multimodal data, giving due consideration to the multiplicity of theoretical frameworks and theories which have been developed for the study of multimodal text within social semiotics, and their impact on the development of a computer-based tool for the exploration, annotation, and analysis of multimodal data.
Color separation and highly optimized context tree modeling for binary layers have provided the best compression results for color map images that consist of highly complex spatial structures but only a relatively few number of colors. We explore whether this kind of approach works on photographic and palette images as well. The main difficulty is that these images can have a much higher number of colors, and it is therefore much more difficult to exploit spatial dependencies via binary layers. The original contributions of this work include: 1. the application of context-tree-based compression (previously designed for map images) to natural and color palette images; 2. the consideration of four different methods for bit-plane separation; and 3. Extension of the two-layer context to a multilayer context for better utilization of the crosslayer correlations. The proposed combination is extensively compared to state of the art lossless image compression methods.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.