Improving Sequence Modeling Ability of Recurrent Neural Networks via Sememes

Qin, Yujia; Qi, Fanchao; Ouyang, Sicong; Liu, Zhiyuan; Yang, Cheng; Wang, Yasheng; Li, Qun; Sun, Maosong

doi:10.1109/taslp.2020.3012060

Cited by 18 publications

(23 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use the official implementation of SememeCell (Qin et al, 2020) embedding size, we choose the medium version of BERT, which has 512-dimensional hidden vectors and 8 layers. 6 As for evaluation metrics, we use accuracy for both NLI and sentiment analysis.…”

Section: Experimental Settingsmentioning

confidence: 99%

“…When deep learning becomes the mainstream approach of NLP, the usefulness of HowNet is also proved in diverse tasks including word representation learning (Sun and Chen, 2016;Niu et al, 2017), language modeling (Gu et al, 2018), semantic composition (Qi et al, 2019a), sequence modeling (Qin et al, 2020), reverse dictionary , word sense disambiguation (Hou et al, 2020), textual adversarial attacking and backdoor attacking (Qi et al, 2021).…”

Section: Hownet and Its Applicationsmentioning

confidence: 99%

“…But negators like "not" are retained because they are critical to the meanings of words. In addition, according to previous work Qin et al, 2020), sememes that are annotated to too many or too few words are usually uninformative and ineffective to downstream applications. Therefore, we count the frequencies of words in the CDV occurring in all definitions and empirically remove the most frequent 1% and the infrequent 10%.…”

Section: Constructing the Sememe Setmentioning

confidence: 99%

“…First, the sememe-to-word semantic compositionality endows HowNet with particular suitability for integration into neural networks (Qi et al, 2019a;Li et al, 2019). The sememes of a word can be re-garded as semantic labels and easily incorporated into the neural processing unit of the word, e.g., a cell of RNN (Qin et al, 2020). Second, the nature that a limited set of sememes are used to express meanings of unlimited words makes HowNet very useful in low-data regimes, e.g., improving embeddings of rare words (Sun and Chen, 2016;Niu et al, 2017), where sememes serve as a bridge between high-frequency and rare words.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Automatic Construction of Sememe Knowledge Bases via Dictionaries

Qi¹,

Chen²,

Feng-yu³

et al. 2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

Self Cite

View full text Add to dashboard Cite

A sememe is defined as the minimum semantic unit in linguistics. Sememe knowledge bases (SKBs), which comprise words annotated with sememes, enable sememes to be applied to natural language processing. So far a large body of research has showcased the unique advantages and effectiveness of SKBs in various tasks. However, most languages have no SKBs, and manual construction of SKBs is time-consuming and labor-intensive. To tackle this challenge, we propose a simple and fully automatic method of building an SKB via an existing dictionary. We use this method to build an English SKB and a French SKB, and conduct comprehensive evaluations from both intrinsic and extrinsic perspectives. Experimental results demonstrate that the automatically built English SKB is even superior to HowNet, the most widely used SKB that takes decades to build manually. And both the English and French SKBs can bring obvious performance enhancement in multiple downstream tasks. All the code and data of this paper (except the copyrighted dictionaries) can be obtained at https://github.com/ thunlp/DictSKB.

show abstract

Section: Experimental Settingsmentioning

confidence: 99%

Section: Hownet and Its Applicationsmentioning

confidence: 99%

Section: Constructing the Sememe Setmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Automatic Construction of Sememe Knowledge Bases via Dictionaries

Qi¹,

Chen²,

Feng-yu³

et al. 2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…It pre-defines a set of about 2, 000 sememes and uses them to annotate senses of more than 100, 000 Chinese words and phrases. In recent years, HowNet has been successfully applied to diverse natural language processing tasks such as language modeling (Gu et al, 2018), semantic composition (Qi et al, 2019a), sequence modeling (Qin et al, 2020), textual adversarial attack (Zang et al, 2020) and reverse dictionary .…”

Section: Introduction To Hownetmentioning

confidence: 99%

Try to Substitute: An Unsupervised Chinese Word Sense Disambiguation Method Based on HowNet

Bairu¹,

Qi²,

Zang³

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

Self Cite

View full text Add to dashboard Cite

Word sense disambiguation (WSD) is a fundamental natural language processing task. Unsupervised knowledge-based WSD only relies on a lexical knowledge base as the sense inventory and has wider practical use than supervised WSD that requires a mass of sense-annotated data. HowNet is the most widely used lexical knowledge base in Chinese WSD. Because of its uniqueness, however, most of existing unsupervised WSD methods cannot work for HowNetbased WSD, and the tailor-made methods have not obtained satisfying results. In this paper, we propose a new unsupervised method for HowNet-based Chinese WSD, which exploits the masked language model task of pre-trained language models. In experiments, considering existing evaluation dataset is small and out-of-date, we build a new and larger HowNet-based WSD dataset. Experimental results demonstrate that our model achieves significantly better performance than all the baseline methods. All the code and data of this paper are available at https://github.com/thunlp/SememeWSD.

show abstract