A method of Named Entity Recognition in Classical Chinese based on Bert-Ancient-Chinese

Feng, Ping; Kang, Jae-Won; Huang, Biao; Wang, Shudan; Li, Jialun

doi:10.1109/mlbdbi58171.2022.00033

Cited by 2 publications

(1 citation statement)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhang et al [21] employed augmentation strategies, including continual pre-training, adversarial training and ensemble learning, to improve the performance of language models on word segmentation and part-of-speech tagging in ancient Chinese. Feng et al [22] designed a model architecture, Bert-ancient-chinese+LSTM+CRF (BAC+RLSTM+ CRF) for named entity recognition in ancient Chinese. Wang et al [23] created a model that included a multi-head attention layer to capture long-distance related features, by extracting entities and their relationships simultaneously.…”

Section: Related Workmentioning

confidence: 99%

A complex network approach to analyse pre-trained language models for ancient Chinese

Zheng,

Xiao

2024

R. Soc. Open Sci.

View full text Add to dashboard Cite

Ancient Chinese is a splendid treasure within Chinese culture. To facilitate its compilation, pre-trained language models for ancient Chinese are developed. After that, researchers are actively exploring the factors contributing to their success. However, previous work did not study how language models organized the elements of ancient Chinese from a holistic perspective. Hence, we adopt complex networks to explore how language models organize the elements in ancient Chinese system. Specifically, we first analyse the characters’ and words’ co-occurrence networks in ancient Chinese. Then, we study characters’ and words’ attention networks, generated by attention heads within SikuBERT from two aspects: static and dynamic network analysis. In the static network analysis, we find that (i) most of attention networks exhibit small-world properties and scale-free behaviour, (ii) over 80% of attention networks exhibit high similarity with the corresponding co-occurrence networks, (iii) there exists a noticeable gap between characters’ and words’ attention networks across layers, while their fluctuations remain relatively consistent, and (iv) the attention networks generated by SikuBERT tend to be sparser compared with those from Chinese BERT. In dynamic network analysis, we find that the sentence segmentation task does not significantly affect network metrics, while the part-of-speech tagging task makes attention networks sparser.

show abstract

Section: Related Workmentioning

confidence: 99%

A complex network approach to analyse pre-trained language models for ancient Chinese

Zheng,

Xiao

2024

R. Soc. Open Sci.

View full text Add to dashboard Cite

show abstract

Semantic-enhanced graph neural network for named entity recognition in ancient Chinese books

Xu,

Mao,

Wang

et al. 2024

Sci Rep

View full text Add to dashboard Cite

Named entity recognition (NER) plays a crucial role in the extraction and utilization of knowledge of ancient Chinese books. However, the challenges of ancient Chinese NER not only originate from linguistic features such as the use of single characters and short sentences but are also exacerbated by the scarcity of training data. These factors together limit the capability of deep learning models, like BERT-CRF, in capturing the semantic representation of ancient Chinese characters. In this paper, we explore the semantic enhancement of NER in ancient Chinese books through the utilization of external knowledge. We propose a novel model based on Graph Neural Networks that integrates two different forms of external knowledge: dictionary-level and chapter-level information. Through the Graph Attention Mechanism (GAT), these external knowledge are effectively incorporated into the model’s input context. Our model is evaluated on the C_CLUE dataset, showing an improvement of 3.82% over the baseline BAC-CRF model. It also achieves the best score compared to several state-of-the-art dictionary-augmented models.

show abstract

A method of Named Entity Recognition in Classical Chinese based on Bert-Ancient-Chinese

Cited by 2 publications

References 2 publications

A complex network approach to analyse pre-trained language models for ancient Chinese

A complex network approach to analyse pre-trained language models for ancient Chinese

Semantic-enhanced graph neural network for named entity recognition in ancient Chinese books

Contact Info

Product

Resources

About