A survey on syntactic processing techniques

Zhang, Xulang; Mao, Rui; Wang, Zhaoxia

doi:10.1007/s10462-022-10300-7

Cited by 7 publications

(4 citation statements)

References 196 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, there has been limited research on how syntactic tasks of different granularity affect each other. In traditional feature-engineering-based approaches, PoS tags are commonly used as input features for coarser-grained syntactic tasks including SBD and text chunking [24]. Modern neural-network-based techniques such as multi-task learning (MTL) [11,25] and transfer learning [26] also show that PoS tagging is a complimentary task for text chunking, but the reverse is inconclusive.…”

Section: Introductionmentioning

confidence: 99%

Granular Syntax Processing with Multi-task and Curriculum Learning

Zhang,

Mao,

Cambria

2023

Preprint

View full text Add to dashboard Cite

Syntactic processing techniques are the foundation of Natural Language Processing (NLP), supporting many downstream NLP tasks. In this paper, we conduct pair-wise Multi-Task Learning (MTL) on syntactic tasks with different granularity, namely Sentence Boundary Detection (SBD), text chunking, and Part-of-Speech(PoS) tagging, so as to investigate the extent to which they complement each other. We propose a novel soft parameter sharing mechanism to share local and global dependency information that is learned from both target tasks. We also propose a Curriculum Learning (CL) mechanism to improve MTL with non-parallel labeled data. Using non-parallel labeled data in MTL is a common practice, whereas it has not received enough attention before. For example, our employed PoS tagging data do not have text chunking labels. When learning PoS tagging and text chunking together, the proposed CL mechanism aims to select complementary samples from the two tasks to update the parameters of the MTL model in the same training batch. Such a method yields better performance and learning stability. We conclude that the fine-grained tasks can provide complementary features to coarse-grained ones, while the most coarse-grained task, SBD, provides useful information for the most fine-grained one, PoS tagging. Additionally, the text chunking task achieves state-of-the-art performance when joint learning with PoS tagging. Our analytical experiments also show the effectiveness of the proposed soft parameter sharing and CL mechanisms.

show abstract

Section: Introductionmentioning

confidence: 99%

Granular Syntax Processing with Multi-task and Curriculum Learning

Zhang,

Mao,

Cambria

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Semantics is a linguistic term, generally referring to the meaning of language. Unlike syntax which studies the structure of sentences [395], the significance of semantics lies in its ability to aid our comprehension of how meaning is conveyed through words, phrases, and sentences, as well as how language is used to express various ideas, thoughts, and emotions. The semantic processing techniques we include in this section are WSD and Pre-trained Language Model (PLM).…”

Section: Semantic Processingmentioning

confidence: 99%

“…However, there has been limited research on how syntactic tasks of different granularity affect each other. In traditional feature-engineering-based approaches, POS tags are commonly used as input features for coarser-grained syntactic tasks including SBD and text chunking [394]. Modern neural-network-based techniques such as MTL [74,84] and transfer learning [387] also show that POS tagging is a complimentary task for text chunking, but the reverse is inconclusive.…”

Section: Introductionmentioning

confidence: 99%

“…In the field of NLP, the need for deep neural networks is usually due to the complexity of modeling language. For example, syntax [394] and semantics [212] will exhibit different patterns in different contexts. As a result, researchers achieve the learning of complex word sequence patterns by making the model deeper and adding different learning mechanisms, e.g., the gating-mechanismbased Long Short-Term Memory [142] and the attention-based Transformer [361].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Disentangling syntactics, semantics, and pragmatics in natural language processing

Zhang

View full text Add to dashboard Cite

It has been my utmost privilege to be given this opportunity to pursue a Ph.D degree.First, I would like to express my sincere gratitude to my supervisor, Prof. Erik Cambria for his support and guidance throughout my Ph.D candidature. It is with his warm encouragement that I found the courage and confidence to pursue a Ph.D degree.Next, I would like to thank my co-supervisor, Dr Rui Mao, for his patient guidance.His insights and advice played a vital role in orientating my research direction and raising the quality of my works. I deeply appreciate his efforts in helping me with my Ph.D project.I would also like to thank my fellow Sentic Team members Mengshi Ge, Luyao Zhu, Jinjie Ni, Zonglin Yang, Kai He, Tom Yang, etc., for their friendship.Last but not the least, I owe enormous gratitude to my parents. The achievements in this thesis would not have come to be if not for their love, care, and encouragement. Special thanks goes to my long-time friends Wenrui Li, Qiyu Zhong, and Zhixin Xu, who lifted my spirit in times of stress and anxiety with compassion and killer memes.

show abstract

Exploring the Integration of Neural Networks in Quality-Oriented Education for Undergraduates: Evolutionary Computing-Based Personalized Learning Path Recommendation

Dong,

Zhao

2024

Lecture Notes in Electrical Engineering

View full text Add to dashboard Cite

A survey on syntactic processing techniques

Cited by 7 publications

References 196 publications

Granular Syntax Processing with Multi-task and Curriculum Learning

Granular Syntax Processing with Multi-task and Curriculum Learning

Disentangling syntactics, semantics, and pragmatics in natural language processing

Exploring the Integration of Neural Networks in Quality-Oriented Education for Undergraduates: Evolutionary Computing-Based Personalized Learning Path Recommendation

Contact Info

Product

Resources

About