Development and preliminary evaluation of the MIT ATIS system

Abstract

This paper represents a status report on the MIT ATIS system. The most significant new achievement is that we now have a speech-input mode. It is based on the MIT SUMMIT system using context independent phone models, and includes a word-pair grammar with perplexity 92 (on the June-90 test set). In addition, we have completely redesigned the back-end component, in order to emphasize portability and extensibility. The parser now produces an intermediate semantic frame representation, which serves as the focal point for all back-end operations, such as history management, text generation, and SQL query generation. Most of those aspects of the system that are tied to a particular domain are now entered through a set of tables associated with a small artificial language for decoding them. We have also improved the display of the database table, making it considerably easier for a subject to comprehend the information given. We report here on the results of the official DARPA February-91 evaluation, as well as on results of an evaluation on data collected at MIT, for both speech input and text input.

PEGASUS: A Spoken Language Interface for On-Line Air Travel Planning

“…Figure 1 shows a block diagram of PEGASUS. The speech understanding component makes use of much of the original ATIS system to process user queries [5,6]. The segment-based SUMMIT speech recognition component [7] produces a list of the top ten sentence hypotheses, which are then filtered by the probabilistic TINA natural language component [8].…”

A Bilingual VOYAGER System

“…We first converted the system so that it could generate responses in Japanese. This enabled us to collect data from native speakers of Japanese in a wizard mode whereby an experimentor would translate the subjects' spoken input and type the resulting English queries to the system [3,9]. Once data were available we were able to port the speech recognition and language understanding components.…”

“…The VOYAGER application uses an object-oriented database, although we have also accessed data in SQL and other configurations [3]. Responses to the user consist of displays, text, and synthetic speech.…”

“…In a manner similar to data collection techniques used for the ATIS domain [3], subjects were asked to solve four problem scenarios. At the end of the session subjects were also allowed to ask random questions of the system.…”

“…While VOYAGER is constrained both in its capabilities and domain of knowledge, it contains all the essential components of a spoken-language system, including discourse maintenance and language generation. The VOYAGER application provided us with our first experience with the development of spoken language systems, helped us understand the issues related to this endeavor, and provided a framework for our subsequent system development efforts [3,4]. Over the past few years, we have become increasingly interested in developing multilingual spoken language systems.…”