Language-Picture Question-Answering Through Common Semantic Representation And its Application to the World of Weather Report

Yokota, Masao; Taniguchi, Rin-ichiro; Kawaguchi, Eiji

doi:10.1007/978-3-642-82285-8_3

Cited by 7 publications

(5 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The most important problems remaining unsolved are how to provide each attribute space and how to build its corresponding atomic performance. These problems concern neuroscience (Brooks, 1986) as well as psychology (Yokota, 1988;Fukuda et al, 1998;Sugita et al, 2003) and therefore the author will consider employment of soft computing theories such as neural network, genetic algorithm, fuzzy logic, etc. for their self-organization in the near future.…”

Section: Conlusionmentioning

confidence: 99%

“…Here, 'integrated multimedia understanding' means especially 'multimedia understanding based on such a knowledge representation common to multiple information media (i.e., natural language, picture, music, gesture, etc.) that can facilitate cross-media operations (Yokota et al, 1984;Eakins & Graham, 1999;Kherfi et al, 2004;Yokota & Capi, 2005a)'. For ordinary people, however, natural language is the most important because it can convey the exact intention of the sender to the receiver due to its syntax and semantics common to its users, which is not necessarily the www.intechopen.com Humanoid Robots 334 case for another information medium such as gesture or so.…”

Section: Introductionmentioning

confidence: 99%

“…More analytically, these omnisensory images are associated with spatiotemporal changes (or constancies) in certain attributes of the matters scanned by FAO and modeled as temporally parameterized "loci in attribute spaces", so www.intechopen.com Towards Artificial Communication Partners with a Multiagent Mind Model Based on Mental Image Directed Semantic Theory 335 called, to be formulated in the formal language L md . This language has already been implemented on several types of computerized intelligent systems including IMAGES-M (e.g., Yokota et al, 1984;Oda, et al, 2001;Amano, et al, 2005;Yokota & Capi, 2005a). The most remarkable feature of L md is its capability of formalizing spatiotemporal matter concepts grounded in human/robotic sensation while the other similar KRLs are designed to describe the logical relations among conceptual primitives represented by lexical tokens (e.g., Dorr & Bonnie, 1997;Zarri, 1997;Sowa, 2000).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Towards Artificial Communication Partners with a Multiagent Mind Model Based on Mental Image Directed Semantic Theory

Yokota¹

2009

Humanoid Robots

Self Cite

View full text Add to dashboard Cite

Section: Conlusionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Towards Artificial Communication Partners with a Multiagent Mind Model Based on Mental Image Directed Semantic Theory

Yokota¹

2009

Humanoid Robots

Self Cite

View full text Add to dashboard Cite

“…With image data, content analysis of the data can be combined with natural-language processing of the captions to supplement the original English caption, as in the weather map system of Yokokota et al [1984] and the face-locating program of Srihari and Rapaport [1989]. But content analysis of audio, pictures, and video can be very slow and is only successful for narrowly restricted content.…”

Section: Previous Workmentioning

confidence: 99%

Natural-language retrieval of images based on descriptive captions

Guglielmo

Rowe

1996

ACM Trans. Inf. Syst.

View full text Add to dashboard Cite

We describe a prototype intelligent information retrieval system that uses natural-language understanding to efficiently locate captioned data. Multimedia data generally require captions to explain their features and significance. Such descriptive captions often rely on long nominal compounds (strings of consecutive nouns) which create problems of disambiguating word sense. In our system, captions and user queries are parsed and interpreted to produce a logical form, using a detailed theory of the meaning of nominal compounds. A fine-grain match can then compare the logical form of the query to the logical forms for each caption. To improve system efficiency, we first perform a coarse-grain match with index files, using nouns and verbs extracted from the query. Our experiments with randomly selected queries and captions from an existing image library show an increase of 30% in precision and 50% in recall over the keyphrase approach currently used. Our processing times have a median of seven seconds as compared to eight minutes for the existing system, and our system is much easier to use.

show abstract

“…Although not of immediate practical value, they provide new insight and enrich the methodology. Yokota et al (1984) studied the relations between visual and textual presentations of weather reports and the translation of forecasts from one form to another. They offered various techniques for understanding and generating weather messages in graphical and textual form.…”

Section: B Experimental Systems For Weather Report Manipulationmentioning

confidence: 99%

Model-Driven, Assertion-Based Generation of Multimedia Weather Information

Kerpedjiev¹

1995

Bull. Amer. Meteor. Soc.

View full text Add to dashboard Cite

As gridded forecasts are becoming a common source of weather data, efforts are being made to automatically produce displays for various categories of decision makers such as emergency managers and road traffic officers. This paper proposes a new systematic approach to the generation of multimedia weather reports tailored to the specific needs of the user. The process consists of two steps: extraction of assertions from gridded datasets, and their presentation in the form of maps, charts, tables, and natural language text. The assertions organized by discourse relations relate weather elements to regions and periods of interest to the user. The user can explore the situation in a hypermedia mode by descending from summary surveillance displays to more detailed exploratory ones. This approach provides a structure for a class of applications dealing with the dissemination of weather information and supports them with general methods of assertion extraction and media generation. The application-specific knowledge is localized in five orthogonal and easily maintained models: territory, time, parameters, phenomena, and events. A prototype system has demonstrated the viability of the concepts.

show abstract

Language-Picture Question-Answering Through Common Semantic Representation And its Application to the World of Weather Report

Cited by 7 publications

References 4 publications

Towards Artificial Communication Partners with a Multiagent Mind Model Based on Mental Image Directed Semantic Theory

Towards Artificial Communication Partners with a Multiagent Mind Model Based on Mental Image Directed Semantic Theory

Natural-language retrieval of images based on descriptive captions

Model-Driven, Assertion-Based Generation of Multimedia Weather Information

Contact Info

Product

Resources

About