Predicting Entity Popularity to Improve Spoken Entity Recognition by Virtual Assistants

Gysel, C; Tsagkias, Manos; Pusateri, Ernest; Oparin, Ilya

doi:10.1145/3397271.3401298

Cited by 6 publications

(3 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…System description. Our ASR system consists of an acoustic model that is a deep convolutional neural network [14], a 4-gram LM with Good-Turing smoothing in the first pass (see [15,16] for details), and the same LM interpolated with a Feed-Forward Neural Network (FFNN) LM [17] in the second pass. To build a scalable TTS-ASR loop, we use our previous generation speech synthesizer, a unit selection system described in [18].…”

Section: Methodsmentioning

confidence: 99%

Error-Driven Pruning of Language Models for Virtual Assistants

Gondala

Verwimp

Pusateri

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Language models (LMs) for virtual assistants (VAs) are typically trained on large amounts of data, resulting in prohibitively large models which require excessive memory and/or cannot be used to serve user requests in real-time. Entropy pruning results in smaller models but with significant degradation of effectiveness in the tail of the user request distribution. We customize entropy pruning by allowing for a keep list of infrequent n-grams that require a more relaxed pruning threshold, and propose three methods to construct the keep list. Each method has its own advantages and disadvantages with respect to LM size, ASR accuracy and cost of constructing the keep list. Our best LM gives 8% average Word Error Rate (WER) reduction on a targeted test set, but is 3 times larger than the baseline. We also propose discriminative methods to reduce the size of the LM while retaining the majority of the WER gains achieved by the largest LM.

show abstract

Section: Methodsmentioning

confidence: 99%

Error-Driven Pruning of Language Models for Virtual Assistants

Gondala

Verwimp

Pusateri

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…With the integration of language models in voice assistants, users can interact with systems using natural language. They can provide flexibility in user queries for different language use, such as synonyms, and alternative phrasings, and can compensate for inaccurate voice transcription due to the prerecorded priors [68]. This capacity is attributed to LLMs' ability to comprehend intentions and generate natural language in a contextualized manner.…”

Section: Large Language Models In Virtual Assistantsmentioning

confidence: 99%

Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory Augmentation

Zulfikar,

Chan,

Maes

2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

People have to remember an ever-expanding volume of information. Wearables that use information capture and retrieval for memory augmentation can help but can be disruptive and cumbersome in real-world tasks, such as in social settings. To address this, we developed Memoro, a wearable audio-based memory assistant with a concise user interface. Memoro uses a large language model (LLM) to infer the user's memory needs in a conversational context, semantically search memories, and present minimal suggestions. The assistant has two interaction modes: Query Mode for voicing queries and Queryless Mode for on-demand predictive assistance, without explicit query. Our study of (N=20) participants engaged in a realtime conversation, demonstrated that using Memoro reduced device Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).

show abstract

“…High-quality automatic speech recognition (ASR) is essential for these systems to work well. However, an important subset of requests, those containing named entities, still presents a significant challenge to ASR [2]. This is primarily because it is difficult to build a language model (LM) that accurately models entities that are less popular or only recently popular, as they show up rarely or not at all in training data.…”

Section: Introductionmentioning

confidence: 99%

A Discriminative Entity-Aware Language Model for Virtual Assistants

Saebi,

Pusateri,

Meghawat

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

High-quality automatic speech recognition (ASR) is essential for virtual assistants (VAs) to work well. However, ASR often performs poorly on VA requests containing named entities. In this work, we start from the observation that many ASR errors on named entities are inconsistent with real-world knowledge. We extend previous discriminative n-gram language modeling approaches to incorporate real-world knowledge from a Knowledge Graph (KG), using features that capture entity type-entity and entity-entity relationships. We apply our model through an efficient lattice rescoring process, achieving relative sentence error rate reductions of more than 25% on some synthesized test sets covering less popular entities, with minimal degradation on a uniformly sampled VA test set.

show abstract

Predicting Entity Popularity to Improve Spoken Entity Recognition by Virtual Assistants

Cited by 6 publications

References 42 publications

Error-Driven Pruning of Language Models for Virtual Assistants

Error-Driven Pruning of Language Models for Virtual Assistants

Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory Augmentation

A Discriminative Entity-Aware Language Model for Virtual Assistants

Contact Info

Product

Resources

About