Oral-Query-by-Sketch: An XML-Based Framework for Speech Access to Image Databases

Abstract

We propose an XML-based framework for speech access to multiple image databases without the need for keyboard or pen-based input. The key idea is to orally command the system in drawing an abstract sketch of the target images using simple graphical objects. We define an oral language for image description and query control named SpeechQuel. The abstract sketch is represented in W3C SVG format while all other query constraints are represented using a device and system independent XML language called SpeechQuelX. This facilitates content-based access to multiple and/or heterogeneous image databases with a single query. Preliminary implementation result successfully demonstrates the feasibility and efficiency of our approach.