Hierarchical Character Embeddings: Learning Phonological and Semantic Representations in Languages of Logographic Origin Using Recursive Neural Networks

Nguyen, Minh; Ngo, Gia H.; Chen, Nancy F.

doi:10.1109/taslp.2019.2955246

Cited by 17 publications

(9 citation statements)

References 49 publications

(70 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where u i denotes the i-th word in the target language sequence and c i denotes the background vector of word i. Since the background vectors of the LSTM model with the embedded attention mechanism are a set of multiple vectors, rather than being uniformly fixed [19], each word in the target language sequence can find a unique background vector corresponding to it.…”

Section: Models In Ismentioning

confidence: 99%

LSTM-Based Attentional Embedding for English Machine Translation

Jian

Xiang

2022

Scientific Programming

View full text Add to dashboard Cite

In order to reduce the workload of manual grading and improve the efficiency of grading, a computerized intelligent grading system for English translation based on natural language processing is designed. An attention-embedded LSTM English machine translation model is proposed. Firstly, according to the characteristics of the standard LSTM network model that uses fixed dimensional vectors to represent words in the encoding stage, an English machine translation model based on LSTM attention embedding is established; the structure level of the English translation scoring system is constructed. A linguistic model of the English translation scoring system is established, and the probability distribution of a particular sentence sequence or word sequence of the translated text is statistically calculated using the model. The results show that the English machine translation model based on LSTM attention embedding proposed in this study can enhance the representation of the source language contextual information and improve the performance of the English machine translation model and the quality of the translation compared with the English machine translation models constructed by existing neural network structures, such as standard LSTM models, RNN models, and GRU-Attention translation models.

show abstract

Section: Models In Ismentioning

confidence: 99%

LSTM-Based Attentional Embedding for English Machine Translation

Jian

Xiang

2022

Scientific Programming

View full text Add to dashboard Cite

show abstract

“…First, we use the file 1 about structures of Han Ideographs and refer to Ke and Hagiwara 2 to get all the Chinese character trees. Then, we use the depth-first algorithm to convert each character tree into the format of a sequence (Nguyen et al, 2019). Note that, there are two types of tokens in the input sequence.…”

Section: Pre-training Model Architecturementioning

confidence: 99%

Glyph Enhanced Chinese Character Pre-Training for Lexical Sememe Prediction

Lyu¹,

Chen²,

Yu³

2021

Findings of the Association for Computational Linguistics: EMNLP 2021

View full text Add to dashboard Cite

Sememes are defined as the atomic units to describe the semantic meaning of concepts. Due to the difficulty of manually annotating sememes and the inconsistency of annotations between experts, the lexical sememe prediction task has been proposed. However, previous methods heavily rely on word or character embeddings, and ignore the fine-grained information. In this paper, we propose a novel pre-training method which is designed to better incorporate the internal information of Chinese character. The Glyph enhanced Chinese Character representation (GCC) is used to assist sememe prediction. We experiment and evaluate our model on HowNet, which is a famous sememe knowledge base. The experimental results show that our method outperforms existing non-external information models.

show abstract

“…Such an encoding has been applied for semantic or sentiment classification [26][27][28], named-entity recognition [29], and has also been used in neural machine translation [32,33]. Moreover, encoding of subwordunit-based tree structures using tree-RNN [30,31] has also been studied for better word representations.…”

Section: Tree-rnnmentioning

confidence: 99%

Tree-Constrained Pointer Generator for End-to-End Contextual Speech Recognition

Sun

Zhang

Woodland

2021

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

View full text Add to dashboard Cite

Incorporating biasing words obtained as contextual knowledge is critical for many automatic speech recognition (ASR) applications. This paper proposes the use of graph neural network (GNN) encodings in a tree-constrained pointer generator (TCP-Gen) component for end-to-end contextual ASR. By encoding the biasing words in the prefix-tree with a tree-based GNN, lookahead for future wordpieces in end-to-end ASR decoding is achieved at each tree node by incorporating information about all wordpieces on the tree branches rooted from it, which allows a more accurate prediction of the generation probability of the biasing words. Systems were evaluated on the Librispeech corpus using simulated biasing tasks, and on the AMI corpus by proposing a novel visual-grounded contextual ASR pipeline that extracts biasing words from slides alongside each meeting. Results showed that TCPGen with GNN encodings achieved about a further 15% relative WER reduction on the biasing words compared to the original TCPGen, with a negligible increase in the computation cost for decoding.

show abstract

Hierarchical Character Embeddings: Learning Phonological and Semantic Representations in Languages of Logographic Origin Using Recursive Neural Networks

Cited by 17 publications

References 49 publications

LSTM-Based Attentional Embedding for English Machine Translation

LSTM-Based Attentional Embedding for English Machine Translation

Glyph Enhanced Chinese Character Pre-Training for Lexical Sememe Prediction

Tree-Constrained Pointer Generator for End-to-End Contextual Speech Recognition

Contact Info

Product

Resources

About