FROST: Fallback Voice Apps Recommendation for Unhandled Voice Commands in Intelligent Personal Assistants

Abstract

Intelligent personal assistants (IPAs) such as Amazon Alexa, Google Assistant and Apple Siri extend their built-in capabilities by supporting voice apps developed by third-party developers. Sometimes the smart assistant is not able to successfully respond to user voice commands (aka utterances). There are many reasons including automatic speech recognition (ASR) error, natural language understanding (NLU) error, routing utterances to an irrelevant voice app, or simply that the user is asking for a capability that is not supported yet. The failure to handle a voice command leads to customer frustration. In this article, we introduce a fallback skill recommendation system (FROST) to suggest a voice app to a customer for an unhandled voice command. There are several practical issues when developing a skill recommender system for IPAs, i.e., partial observation, hard and noisy utterances. To solve the partial observation problem, we propose collaborative data relabeling (CDR) method. To mitigate hard and noisy utterance issues, we propose a rephrase-based relabeling technique. We evaluate the proposed system in both offline and online settings. The offline evaluation results show that the FROST system outperforms the baseline rule-based system. The online A/B testing results show a significant gain of customer experience metrics.

Urban intelligent assistant on the example of the escalator passenger safety management at the subway stations

“…Recent research has led to significant advancements in modern intelligent assistants, particularly in the realm of voice interaction with users. These advancements have been achieved through the implementation of extended models and sophisticated methods for natural language processing 52 , 61 . Specialized assistants, such as Viv, have been developed, enabling comprehension of complex user queries.…”