Extensions of recurrent neural network language model

Mikolov, Tomáš; Kombrink, Stefan; Burget, Lukáš; Černocký, Jaň; Khudanpur, Sanjeev

doi:10.1109/icassp.2011.5947611

Cited by 3,120 publications

(3,171 citation statements)

References 13 publications

Supporting

Mentioning

3,117

Contrasting

Unclassified

Order By: Relevance

“…Convolution neural networks take advantage of existing local structures and share weights, which can dramatically reduce over-fitting problems that occur in fully connected networks. For the sequence data of acoustics (Sak et al, 2014) and language (Mikolov et al, 2010;Vinyals et al, 2015), models with recurrent structures have brought significant improvements to state-of-the-art performance. They have introduced chain-like loop structures into recurrent neural networks (Funahashi and Nakamura, 1993) as shown in Figs.…”

Section: Trends In the Development Of Ai Technology Applications Formentioning

confidence: 99%

Current trends in the development of intelligent unmanned autonomous systems

Zhang

et al. 2017

Frontiers Inf Technol Electronic Eng

126

View full text Add to dashboard Cite

Intelligent unmanned autonomous systems are some of the most important applications of artificial intelligence (AI). The development of such systems can significantly promote innovation in AI technologies. This paper introduces the trends in the development of intelligent unmanned autonomous systems by summarizing the main achievements in each technological platform. Furthermore, we classify the relevant technologies into seven areas, including AI technologies, unmanned vehicles, unmanned aerial vehicles, service robots, space robots, marine robots, and unmanned workshops/intelligent plants. Current trends and developments in each area are introduced.

show abstract

Section: Trends In the Development Of Ai Technology Applications Formentioning

confidence: 99%

Current trends in the development of intelligent unmanned autonomous systems

Zhang

et al. 2017

Frontiers Inf Technol Electronic Eng

126

View full text Add to dashboard Cite

show abstract

“…The main idea is to understand a word not as a separate entity with no specific relation to other words, but to see words as points in a finite dimensional (separable) metric space. The recurrent neural networks -using the similar principle -were successfully introduced to the field of language modelling by T. Mikolov [5] and have become widely used language modelling technique.…”

Section: Recurrent Neural Network Language Modelmentioning

confidence: 99%

“…In last years, recurrent neural networks (RNN) have attracted attention among other types of language models (LM) caused by their better performance [5] and their ability to learn on a smaller corpus than conventional n-gram models. Nowadays, they are considered as a state-of-the-art, especially the Longshort Term Memory (LSTM) variant.…”

Section: Introductionmentioning

confidence: 99%

On Continuous Space Word Representations as Input of LSTM Language Model

Soutner

Müller

2015

Statistical Language and Speech Processing

View full text Add to dashboard Cite

Abstract. Artificial neural networks have become the state-of-the-art in the task of language modelling whereas Long-Short Term Memory (LSTM) networks seem to be an efficient architecture. The continuous skip-gram and the continuous bag of words (CBOW) are algorithms for learning quality distributed vector representations that are able to capture a large number of syntactic and semantic word relationships. In this paper, we carried out experiments with a combination of these powerful models: the continuous representations of words trained with skipgram/CBOW /GloVe method, word cache expressed as a vector using latent Dirichlet allocation (LDA). These all are used on the input of LSTM network instead of 1-of-N coding traditionally used in language models. The proposed models are tested on Penn Treebank and MALACH corpus.

show abstract

“…In Mikolov et al [26] a recurrent neural network (RNN) [24] is used to build a neural language model. The RNN model encode the context word by word and predict the next word.…”

Section: Word Embedding-based Modelsmentioning

confidence: 99%

Word Embedding-Based Approaches for Measuring Semantic Similarity of Arabic-English Sentences

Nagoudi

Ferrero

Schwab

et al. 2018

Communications in Computer and Information Science

View full text Add to dashboard Cite

Abstract. Semantic Textual Similarity (STS) is an important component in manyNatural Language Processing (NLP) applications, and plays an important role in diverse areas such as information retrieval, machine translation, information extraction and plagiarism detection. In this paper we propose two word embeddingbased approaches devoted to measuring the semantic similarity between ArabicEnglish cross-language sentences. The main idea is to exploit Machine Translation (MT) and an improved word embedding representations in order to capture the syntactic and semantic properties of words. MT is used to translate English sentences into Arabic language in order to apply a classical monolingual comparison. Afterwards, two word embedding-based methods are developed to rate the semantic similarity. Additionally, Words Alignment (WA), Inverse Document Frequency (IDF) and Part-of-Speech (POS) weighting are applied on the examined sentences to support the identification of words that are most descriptive in each sentence. The performances of our approaches are evaluated on a crosslanguage dataset containing more than 2400 Arabic-English pairs of sentence. Moreover, the proposed methods are confirmed through the Pearson correlation between our similarity scores and human ratings.

show abstract

Extensions of recurrent neural network language model

Cited by 3,120 publications

References 13 publications

Current trends in the development of intelligent unmanned autonomous systems

Current trends in the development of intelligent unmanned autonomous systems

On Continuous Space Word Representations as Input of LSTM Language Model

Word Embedding-Based Approaches for Measuring Semantic Similarity of Arabic-English Sentences

Contact Info

Product

Resources

About