Framewise phoneme classification with bidirectional LSTM and other neural network architectures

Graves, Alex; Schmidhuber, Jürgen

doi:10.1016/j.neunet.2005.06.042

Cited by 4,328 publications

(2,279 citation statements)

References 5 publications

Supporting

Mentioning

2,065

Contrasting

Unclassified

Order By: Relevance

“…Furthermore, the complexity of a task, and therefore the number of weights likely to be needed for it, does not necessarily increase with the dimensionality of the data. For example, both the networks described in this paper have less than half the weights than the one dimensional networks we have previously applied to speech recognition [4,3]. For a given task, we have also found that using a multi-directional MDRNN gives better results than a uni-directional MDRNN with the same overall number of weights, as previously demonstrated in one dimension [4].…”

Section: Multi-directional Mdrnnssupporting

confidence: 73%

Multi-dimensional Recurrent Neural Networks

Graves

Fernández

Schmidhuber

2007

Lecture Notes in Computer Science

Self Cite

239

180

View full text Add to dashboard Cite

Abstract. Recurrent neural networks (RNNs) have proved effective at one dimensional sequence learning tasks, such as speech and online handwriting recognition. Some of the properties that make RNNs suitable for such tasks, for example robustness to input warping, and the ability to access contextual information, are also desirable in multi-dimensional domains. However, there has so far been no direct way of applying RNNs to data with more than one spatio-temporal dimension. This paper introduces multi-dimensional recurrent neural networks, thereby extending the potential applicability of RNNs to vision, video processing, medical imaging and many other areas, while avoiding the scaling problems that have plagued other multi-dimensional models. Experimental results are provided for two image segmentation tasks.

show abstract

Section: Multi-directional Mdrnnssupporting

confidence: 73%

Multi-dimensional Recurrent Neural Networks

Graves

Fernández

Schmidhuber

2007

Lecture Notes in Computer Science

Self Cite

239

180

View full text Add to dashboard Cite

show abstract

“…More precisely, three connected handwriting competitions at ICDAR 2009 in three different languages (French, Arab, Farsi) were won by deep LSTM RNNs without any a priori linguistic knowledge, performing simultaneous segmentation and recognition. Compare (Graves and Schmidhuber, 2005;Schmidhuber et al, 2011;Graves et al, 2013;Graves and Jaitly, 2014) (Sec. 5.22).…”

Section: : First Official Competitions Won By Rnns and With Mpcnnsmentioning

confidence: 99%

Deep learning in neural networks: An overview

2015

Self Cite

View full text Add to dashboard Cite

In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarises relevant work, much of it from the previous millennium. Shallow and deep learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks.LATEX source: http://www.idsia.ch/˜juergen/DeepLearning8Oct2014.tex Complete BIBTEX file (888 kB): http://www.idsia.ch/˜juergen/deep.bib Preface This is the preprint of an invited Deep Learning (DL) overview. One of its goals is to assign credit to those who contributed to the present state of the art. I acknowledge the limitations of attempting to achieve this goal. The DL research community itself may be viewed as a continually evolving, deep network of scientists who have influenced each other in complex ways. Starting from recent DL results, I tried to trace back the origins of relevant ideas through the past half century and beyond, sometimes using "local search" to follow citations of citations backwards in time. Since not all DL publications properly acknowledge earlier relevant work, additional global search strategies were employed, aided by consulting numerous neural network experts. As a result, the present preprint mostly consists of references. Nevertheless, through an expert selection bias I may have missed important work. A related bias was surely introduced by my special familiarity with the work of my own DL research group in the past quarter-century. For these reasons, this work should be viewed as merely a snapshot of an ongoing credit assignment process. To help improve it, please do not hesitate to send corrections and suggestions to juergen@idsia.ch.

show abstract

“…As the reversed order also takes useful information, a backward representation can be achieved by feeding LSTM with the same input in reverse. We adopt the concatenation of the forward and backward LSTMs outputs, referred to as bidirectional LSTM (Graves and Schmidhuber, 2005). Figure 3a shows the neural network architecture of our E-E, E-T classifier.…”

Section: Temporal Relation Classifiersmentioning

confidence: 99%

Classifying Temporal Relations by Bidirectional LSTM over Dependency Paths

Cheng¹,

Miyao²

2017

Proceedings of the 55th Annual Meeting of the Association For Computational Linguistics (Volume 2: Short Papers)

View full text Add to dashboard Cite

Temporal relation classification is becoming an active research field. Lots of methods have been proposed, while most of them focus on extracting features from external resources. Less attention has been paid to a significant advance in a closely related task: relation extraction. In this work, we borrow a state-of-the-art method in relation extraction by adopting bidirectional long short-term memory (Bi-LSTM) along dependency paths (DP). We make a "common root" assumption to extend DP representations of cross-sentence links. In the final comparison to two stateof-the-art systems on TimeBank-Dense, our model achieves comparable performance, without using external knowledge and manually annotated attributes of entities (class, tense, polarity, etc.).

show abstract

Framewise phoneme classification with bidirectional LSTM and other neural network architectures

Cited by 4,328 publications

References 5 publications

Multi-dimensional Recurrent Neural Networks

Multi-dimensional Recurrent Neural Networks

Deep learning in neural networks: An overview

Classifying Temporal Relations by Bidirectional LSTM over Dependency Paths

Contact Info

Product

Resources

About