Augmented role filling capabilities for semantic interpretation of spoken language

Abstract

This paper describes recent work on the Unisys ATIS Spoken Language System, and reports benchmark results on natural language, spoken language, and speech recognition. We describe enhancements to the system's semantic processing for handling non.transparent argument structure and enhancements to the system's pragmatic processing of material in art. swers displayed to the user. We found that the system's score on the natural language benchmark test decreased from ~8~o to 36~ without these enhancements. We also report results for three spoken language systems, Unisys natural language coupled with MIT-Summit speech recognition, Unisys natural language coupled wish MIT-Lincoln Labs speech recognition and Unisys natural language coupled with BBN speech recognition. Speech recognition results are reported on the results of the Unisys natural language selecting a candidate from the MIT-Summit N-best (N=16).

Session 2

“…(6) Recent work on the Unisys ATIS Spoken Language System was described by Norton et al [11]. "Enhancements to the system's semantic processing for handling nontransparent argument structure and enhancements to the system's pragmatic processing of material in answers displayed to the user" are described.…”

“…The McNemar sentence-error-level significance test (not shown) indicates that the system with the lowest reported word and sentence error rate for the Class A utterances (sys24-a, the Unisys implementation of syntactic, semantic and pragmatic constraints in selecting the first candidate from an N-best listing provided by BBN, described in [11]) has an error rate that is significantly less than all but two other systems, (sysl8-a, the BBN "augmented training" system, and sys06-a, the SRI system). Performance differences (at the sentence error level) between these three systems, however, are not significant.…”

“…Comparisons of results for similar systems for the two larger test subsets (i.e., Class A results vs. Class D1 results) suggest that the Class D1 material is somewhat more difficult to recognize (i.e., the error rates are higher). An interesting hypothesis that may account, in part, for this phenomenon is offered by Norton et al: "...this higher error rate in context dependent spontaneous utterances may be due in part to the presence of prosodic phenomena common in dialogue such as destressing 'old' information" [11].…”

A portable approach to last resort parsing and interpretation

“…Our second domain is called ATIS (Air Travel Information System) [12,13,11]. This is basically a database query application.…”

Recent improvements and benchmark results for the Paramax ATIS system

“…We previously described [1] a feature of the PUN-DIT natural language processing system whereby the *This paper was supported by DARPA contract N000014-89-C0171, administered by the Office of Naval Research, and by internal funding from Paramax Systems Corporation (formerly Unisys Defense Systems). We wish to thank Suzanne Taylor for helpful discussions on applying natural language constraints to OCR enhancement.…”