No abstract

Abstract.To increase the interest and engagement of middle school students in science and technology, the InterFaces project has created virtual museum guides that are in use at the Museum of Science, Boston. The characters use natural language interaction and have near photoreal appearance to increase and presents reports from museum staff on visitor reaction.

The Virtual Museum Guides [1] are two virtual humans set in an exhibit at the Museum of Science, Boston, designed to promote interest in Science, Technology, Engineering and Mathematics (STEM). The primary audience is children between ages 7 to 14, in particular females and other groups under-represented in STEM.The Guides are based on and extend the approach used in the SGT Star character [2] and the Gunslinger project [3]. To interact with the characters, an operator presses a push-totalk button and speaks into a microphone. An audio acquisition client then sends audio to the automatic speech recognizer (ASR), which creates hypotheses of the words that were said, and then sends this text to the Language Understanding (LU) module. The ASR module uses the SONIC toolkit [4], with custom language and acoustic models. The LU module uses a statistical text classification algorithm to map the utterance text onto character responses. It selects a set of responses it believes to be appropriate to the utterance from a domain-specific library of scripted responses and passes them to the dialogue management (DM) module. The DM module uses that response set and the recent dialogue history to select one response for the characters to perform. The response is sometimes a sequence of utterances; in this case, the DM keeps a schedule of pending utterances, and sends them one at a time to the animation components, waiting for a callback signal before sending the next one. If the characters are interrupted by more speech from the operator before the schedule has completed, the DM can cancel the remaining sequence.The LU/DM module pair uses the NPCEditor software ticular word in a character's response given an operator's utterance [6]. When NPCEditor receives a new (possibly unseen) utterance, it uses this translation information to build a model of what it believes to be the best response for the utterance. The classifier then compares this representation to every stored response and returns the best match to the DM part of NPCEditor. In contrast, a traditional text classification approach would compare a new question to sample questions and then directly return the corresponding responses, ignoring the actual text of the response. We have observed that this "translation-based" classification approach significantly increases the effectiveness of the classifier for imperfect speech recognition [6]. NPCEditor has been fielded in a number of applications and has been shown to be successful in noisy classification tasks [5].The Guides have a large but finite set of responses (currently about 400), so the characters might repeat themselves. One of the tasks of the DM is to match the classifier selection to the recent dialogue history and choose responses that have not been heard. The DM also handles cases when the classifier returns no responses. This happens when the operator asks a question for which the characters have no answer or speech is not understood by the ASR module. In that case, the classifier decides that none of the kn...

No abstract