Using proxies for OOV keywords in the keyword search task

Abstract

We propose a simple but effective weighted finite state transducer (WFST) based framework for handling out-ofvocabulary (OOV) keywords in a speech search task. Stateof-the-art large vocabulary continuous speech recognition (LVCSR) and keyword search (KWS) systems are developed for conversational telephone speech in Tagalog. Word-based and phone-based indexes are created from word lattices, the latter by using the LVCSR system's pronunciation lexicon. Pronunciations of OOV keywords are hypothesized via a standard grapheme-to-phoneme method. In-vocabulary proxies (word or phone sequences) are generated for each OOV keyword using WFST techniques that permit incorporation of a phone confusion matrix. Empirical results when searching for the Babel/NIST evaluation keywords in the Babel 10 hour development-test speech collection show that (i) searching for word proxies in the word index significantly outperforms searching for phonetic representations of OOV words in a phone index, and (ii) while phone confusion information yields minor improvement when searching a phone index, it yields up to 40% improvement in actual term weighted value when searching a word index with word proxies.Index Terms-Speech Recognition, Keyword Search, OOV Keywords, Proxy Keywords, Low Resource LVCSR. SEARCHING FOR OOV WORDS IN SPEECHKeyword search (KWS) for spoken documents has become more and more important nowadays as large speech repositories, such as oral history archives [1, 2] and online lectures [3,4] are easily accessible. However, searching for keywords in spoken documents remains a changeling problem. Manual transcription of speech is usually prohibitively expensive, and given the amount of the spoken material available online, it is The authors were supported by DARPA BOLT contract Nō HR0011-12-C-0015, and IARPA BABEL contract Nō W911NF-12-C-0015. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of DARPA, IARPA, DoD/ARL or the U.S. Government. impractical to manually transcribe any nontrivial portion for search. Therefore, automatic KWS is highly desirable. Stateof-the-art KWS systems usually rely on the large vocabulary continuous speech recognition (LVCSR) systems [5,6]. In such systems, lattices of speech segments in the search collection are generated. An inverted index (postings list) is then created from the lattices. KWS may then be performed by searching for a given keyword via the inverted index. The KWS task is ideally "open vocabulary." However, LVCSR systems typically have a fixed vocabulary [7], making it impossible to search for out-of-vocabulary (OOV) words. One therefore uses as large an LVCSR vocabulary as feasible.We are interested in KWS in low resource settings, where only limited resources are available to develop ...

ALBAYZIN 2016 spoken term detection evaluation: an international open competitive evaluation in Spanish

“…3, is the same as the Combined Kaldi system, except that the OOV terms were searched with the proxy words strategy of the Kaldi toolkit [55]. This strategy consists of substituting each OOV word of the search term with acoustically similar INV proxy words, getting rid of the need of a subword-based system for OOV term search.…”

“…The OOV terms were handled using the proxy words strategy of the Kaldi toolkit [55]. As for INV terms, a two-pass strategy has been applied: In the first pass, the OOV terms were synthesized and recognized to create proxy word Finite State Transducers (FSTs) that are needed in the Kaldi term detector to produce a first set of OOV term detections.…”

“…OOV term detection has been carried out using the proxy words mechanism of the Kaldi toolkit [55], as in the Proxy Kaldi system explained before.…”

“…Among these, Kaldi is especially suitable for building STD systems, since this integrates an ASR subsystem, a term detector, and a decision maker [52][53][54]. The Kaldi STD system employs a word-based approach for term detection, and a method based on proxy-words (i.e., replacing each OOV word by the most similar invocabulary word) to detect OOV terms [55].…”

Query-by-example keyword spotting using long short-term memory networks

“…The keywords, however, are only limited to the words that are already defined in the LVCSR vocabulary. While effort has been made to make such systems keywordindependent [11,12,13,14], there is usually a performance degradation when the keywords are out of vocabulary. In addition, these approaches are relatively expensive because of the LVCSR system.…”