The latent words language model

Deschacht, Koen; Belder, Jan De; Moens, Marie‐Francine

doi:10.1016/j.csl.2012.04.001

Cited by 28 publications

(32 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…To further advance towards domain robust ASR, this paper focuses on the latent words LMs (LWLMs) recently proposed in the machine learning area [12]. LWLMs are generative models that have latent variables called latent words.…”

Section: Introductionmentioning

confidence: 99%

N-gram Approximation of Latent Words Language Models for Domain Robust Automatic Speech Recognition

Masumura

Asami

Oba

et al. 2016

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYThis paper aims to improve the domain robustness of language modeling for automatic speech recognition (ASR). To this end, we focus on applying the latent words language model (LWLM) to ASR. LWLMs are generative models whose structure is based on Bayesian soft class-based modeling with vast latent variable space. Their flexible attributes help us to efficiently realize the effects of smoothing and dimensionality reduction and so address the data sparseness problem; LWLMs constructed from limited domain data are expected to robustly cover unknown multiple domains in ASR. However, the attribute flexibility seriously increases computation complexity. If we rigorously compute the generative probability for an observed word sequence, we must consider the huge quantities of all possible latent word assignments. Since this is computationally impractical, some approximation is inevitable for ASR implementation. To solve the problem and apply this approach to ASR, this paper presents an n-gram approximation of LWLM. The n-gram approximation is a method that approximates LWLM as a simple back-off n-gram structure, and offers LWLM-based robust one-pass ASR decoding. Our experiments verify the effectiveness of our approach by evaluating perplexity and ASR performance in not only in-domain data sets but also out-of-domain data sets.

show abstract

Section: Introductionmentioning

confidence: 99%

N-gram Approximation of Latent Words Language Models for Domain Robust Automatic Speech Recognition

Masumura

Asami

Oba

et al. 2016

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

show abstract

“…P (h k |l k , θ) represents the transition probability which can be expressed by an n-gram model for latent words, and P (w k |h k , θ) represents the emission probability that models the dependency between the observed word and the latent word. More details are shown in previous works (Deschacht et al, 2012;Masumura et al, 2013a;Masumura et al, 2013b).…”

Section: Latent Words Language Modelsmentioning

confidence: 99%

“…LWLMs are generative models with single latent word space (Deschacht et al, 2012). The latent word is represented as a specific word that is selected from the entire vocabulary.…”

Section: Latent Words Language Modelsmentioning

confidence: 99%

“…In contrast, latent words LMs (LWLMs) (Deschacht et al, 2012) are clearly effective for outof domain tasks. We employed the LWLM to speech recognition and the resulting performance was significantly superior in out-of domain tasks while the performance was comparable in domainmatched task to conventional LMs (Masumura et al, 2013a;Masumura et al, 2013b).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Hierarchical Latent Words Language Models for Robust Modeling to Out-Of Domain Tasks

Masumura

Asami

Oba

et al. 2015

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

This paper focuses on language modeling with adequate robustness to support different domain tasks. To this end, we propose a hierarchical latent word language model (h-LWLM). The proposed model can be regarded as a generalized form of the standard LWLMs. The key advance is introducing a multiple latent variable space with hierarchical structure. The structure can flexibly take account of linguistic phenomena not present in the training data. This paper details the definition as well as a training method based on layer-wise inference and a practical usage in natural language processing tasks with an approximation technique. Experiments on speech recognition show the effectiveness of h-LWLM in out-of domain tasks.

show abstract

“…For instance, neural networks or Bayesian network models are trained with a large collection of texts to learn distributional semantic models or language models that capture the contexts of a word. As a result, a word can be represented as a vector (i.e., a word embedding) the dimensions of which embed the contextual knowledge (e.g., [43]) or as a predictive language model (e.g., [19]). These works build further on older research of word representations that were learned from large text corpora that predict a word in the context of surrounding words or that learn topical concepts by considering co-occurrences in a full discourse.…”

Section: Current Proposed Solutions: Representation Learning and Deepmentioning

confidence: 99%

Argumentation mining: How can a machine acquire common sense and world knowledge?

Moens

2018

AAC

Self Cite

View full text Add to dashboard Cite

Abstract. Argumentation mining is an advanced form of human language understanding by the machine. This is a challenging task for a machine. When sufficient explicit discourse markers are present in the language utterances, the argumentation can be interpreted by the machine with an acceptable degree of accuracy. However, in many real settings, the mining task is difficult due to the lack or ambiguity of the discourse markers, and the fact that a substantial amount of knowledge needed for the correct recognition of the argumentation, its composing elements and their relationships is not explicitly present in the text, but makes up the background knowledge that humans possess when interpreting language. In this article 1 we focus on how the machine can automatically acquire the needed common sense and world knowledge. As very few research has been done in this respect, many of the ideas proposed in this article are tentative, but start being researched.We give an overview of the latest methods for human language understanding that map language to a formal knowledge representation that facilitates other tasks (for instance, a representation that is used to visualize the argumentation or that is easily shared in a decision or argumentation support system). Most current systems are trained on texts that are manually annotated. Then we go deeper into the new field of representation learning that nowadays is very much studied in computational linguistics. This field investigates methods for representing language as statistical concepts or as vectors, allowing straightforward methods of compositionality. The methods often use deep learning and its underlying neural network technologies to learn concepts from large text collections in an unsupervised way (i.e., without the need for manual annotations). We show how these methods can help the argumentation mining process, but also demonstrate that these methods need further research to automatically acquire the necessary background knowledge and more specifically common sense and world knowledge. We propose a number of ways to improve the learning of common sense and world knowledge by exploiting textual and visual data, and touch upon how we can integrate the learned knowledge in the argumentation mining process.

show abstract

The latent words language model

Cited by 28 publications

References 19 publications

N-gram Approximation of Latent Words Language Models for Domain Robust Automatic Speech Recognition

N-gram Approximation of Latent Words Language Models for Domain Robust Automatic Speech Recognition

Hierarchical Latent Words Language Models for Robust Modeling to Out-Of Domain Tasks

Argumentation mining: How can a machine acquire common sense and world knowledge?

Contact Info

Product

Resources

About