Semantic Role Labeling for Amharic Text Using Multiple Embeddings and Deep Neural Network

Abstract

Amharic is morphologically complex and under-resourced language, posing difficulties in the development of natural language processing applications. This paper presents the development of semantic role labeler for Amharic text using end-to-end deep neural network architecture. The system implicitly captures morphological, semantic and contextual features of a word at different levels of the architecture, and incorporates the syntactic structure of an input sentence. The proposed neural network architecture has four core layers from bottom to top, namely non-contextual word embedding, contextual word embedding, fully connected and sequence decoding layers. The non-contextual word embedding layer is formed from the concatenation of character-based, word-based and sentence-based word embeddings. This layer captures the morphological and semantic features of a given word by making use of BiLSTM recurrent neural network. At the contextual word embedding layer, a context sensitive embedding of a word is generated by applying a new LSTM layer on the top of the non-contextual concatenated word embedding layer. A fully connected network layer is added on top of contextual word embedding layer to supplement it by extracting dependencies among training samples in the corpus. At the sequence decoding layer, a sequence of semantic role labels is predicted using a linear-chain conditional random field algorithm by capturing the dependency among semantic role labels. In addition to the four core layers, the architecture has dropout layers to prevent overfitting problem. The proposed system achieves 94.96% accuracy and 81.2% F1 score when it is tested using test data.

A Systematic Review on Semantic Role Labeling for Information Extraction in Low-Resource Data

“…Text Formality: Formal News A combination of many specific topics English [32], [33], [34], [35], [36], [37], [38], [39], [40]; Catalan, Czech, Spanish [35]; German [35], [37]; Italian [41]; Sinhala [42]; Indonesian [23], [43], [44]; Chinese [35], [45], [46], [47], [37], [39]; Myanmar [48]; Russian [49]; Amharic [50]; Arabic [51], [39]; Persian [52]; Portugese [53]; Vietnamese [54];…”

“…Statistical methods are used to extract patterns or relationships in the data. French [56] News A combination of many specific topics English [40], [35], [37], [38], [85]; Catalan, Czech, Spanish [35]; Indonesian [23], [44]; Amharic [50]; Chinese [35], [46], [47], [37], [85]; German [37], [85]; Italia [ Only one study on SRL that used machine learning was discovered in the last five years. The lack of recent machine learning-based SRL research may be due to the need to identify several features before predicting semantic role labels, which can lead to propagation issues [3].…”

“…Hailu et al [50] developed a deep learning-based SRL method on Amharic text with rich morphology. Thus, character-level input representation becomes an important component in deep learning-based SRL models' combined input representation layers.…”

“…Errors in the parser stage can affect the overall SRL prediction. Hailu et al [50] overcome the complexity and limitations of Amharic lexical resources by using a word embedding and neural network approach to develop an SRL system, where BiLSTM is layered on top of the word embeddings. Devina et al [23] tried to add a highway connection above each BiLSTM layer to get the semantic role of Indonesian news texts.…”

Prediction of Key Parameters of Wheelset Based on LSTM Neural Network

“…The LSTM network is called long short-term memory recurrent neural network [13]. The LSTM neural network also has a chain structure similar to a regular RNN, but it differs in terms of its improved internal structure [14]. Compared to RNN, the LSTM neural network has an additional input and output.…”