Towards a Better Understanding of Noise in Natural Language Processing

Sharou, Khetam Al; Li, Zhenhao; Specia, Lucia

doi:10.26615/978-954-452-072-4_007

Cited by 19 publications

(15 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Two of the identified themes were not replicated in the NLP analysis. This gap in NLP is not unexpected because NLP can miss themes with real-world significance and detect themes that have no contextual relevance (Sharou et al, 2021).…”

Section: Discussionmentioning

confidence: 99%

Entangled: A mixed method analysis of nurses with mental health problems who die by suicide

Barnes

Ayers

et al. 2022

Nursing Inquiry

View full text Add to dashboard Cite

Nurses die by suicide at a higher rate than the general population. Previous studies have observed mental health problems, including substance use, as a prominent antecedent before death. The purpose of this study was to explore the characteristics of nurses who died by suicide documented in the death investigation narratives from the National Violent Death Reporting System from 2003 to 2017 using thematic analysis and natural language processing. One thousand three hundred and fifty-eight subjects met these inclusion criteria. Narratives from 601 subjects were thematically analyzed and 2544 individual narratives were analyzed using natural language processing. The analyses revealed five themes: "mental health treatment," "poor general health and chronic pain," "substance use," "worsening mental health after bereavement," and "repeating a family member's suicide." Mental health/substance use, chronic illness, and chronic pain were seen to coexist in a complex, interdependent manner that appeared to be entangled in the nurses' narratives before death. These findings echo the need for reducing the stigmatization of mental health problems in nursing and removing barriers to help-seeking behaviors as early preventative interventions. Future research is needed to determine if a comprehensive healthcare integration approach to address these entangled problems would reduce suicide vulnerability in nurses and improve their quality of life.

show abstract

Section: Discussionmentioning

confidence: 99%

Entangled: A mixed method analysis of nurses with mental health problems who die by suicide

Barnes

Ayers

et al. 2022

Nursing Inquiry

View full text Add to dashboard Cite

show abstract

“…Based on the previous work on the label noise (Sharou et al, 2021;Larson et al, 2020;Frénay and Verleysen, 2014;Beck et al, 2020), we briefly demonstrate the label noise involved in classification tasks from the following four facets:…”

Section: Analysis Of Label Noisementioning

confidence: 99%

Pre-trained Language Models as Re-Annotators

Chen¹

2022

Preprint

View full text Add to dashboard Cite

Annotation noise are widespread in datasets, but manually revising a flawed corpus is time-consuming and error-prone. Hence, given the prior knowledge in Pre-trained Language Models and the expected uniformity across all annotations, we attempt to reduce annotation noise in the corpus through two tasks automatically: (1) Annotation Inconsistency Detection that indicates the credibility of annotations, and (2) Annotation Error Correction that rectifies the abnormal annotations. We investigate how to acquire semantic sensitive annotation representations from Pre-trained Language Models, expecting to embed the examples with identical annotations to the mutually adjacent positions even without fine-tuning. We proposed a novel credibility score to reveal the likelihood of annotation inconsistencies based on the neighbouring consistency. Then, we fine-tune the Pre-trained Language Models based classifier with cross-validation for annotation correction. The annotation corrector is further elaborated with two approaches: (1) soft labelling by Kernel Density Estimation and (2) a novel distant-peer contrastive loss. We study the re-annotation in relation extraction and create a new manually revised dataset, Re-DocRED, for evaluating document-level re-annotation. The proposed credibility scores show promising agreement with human revisions, achieving a Binary F 1 of 93.4 and 72.5 in detecting inconsistencies on TACRED and DocRED respectively.Moreover, the neighbour-aware classifiers based on distant-peer contrastive learning and uncertain labels achieve Macro F 1 up to 66.2 and 57.8 in correcting annotations on TACRED and DocRED respectively. These improvements are not merely theoretical: Rather, automatically denoised training sets demonstrate up to 3.6% performance improvement for state-of-the-art relation extraction models, and the proposed framework is expected to be hundreds of times faster than the human re-annotators empirically.

show abstract

“…It was pointed out by Al Sharou et al (2021) that textual noise is not always harmful to the system. It could carry a meaning that is important for a certain task.…”

Section: Related Workmentioning

confidence: 99%

“…In this section, we discuss misspelling patterns observed in the corpus and its possible semantics. The term "misspelling" has been generally defined as "when a word is spelt in a way that deviates from reference dictionaries, standardized or accepted norms or recognized usage" (Al Sharou et al, 2021). It includes typos, ad hoc abbreviations, unconventional spellings, phonetic substitutions and lexical deviation.…”

Section: Misspelling Semanticsmentioning

confidence: 99%

Misspelling Semantics In Thai

Nakwijit¹,

Purver²

2022

Preprint

View full text Add to dashboard Cite

User-generated content is full of misspellings. Rather than being just random noise, we hypothesise that many misspellings contain hidden semantics that can be leveraged for language understanding tasks. This paper presents a fine-grained annotated corpus of misspelling in Thai, together with an analysis of misspelling intention and its possible semantics to get a better understanding of the misspelling patterns observed in the corpus. In addition, we introduce two approaches to incorporate the semantics of misspelling: Misspelling Average Embedding (MAE) and Misspelling Semantic Tokens (MST). Experiments on a sentiment analysis task confirm our overall hypothesis: additional semantics from misspelling can boost the micro F1 score up to 0.4-2%, while blindly normalising misspelling is harmful and suboptimal.

show abstract

Towards a Better Understanding of Noise in Natural Language Processing

Cited by 19 publications

References 19 publications

Entangled: A mixed method analysis of nurses with mental health problems who die by suicide

Entangled: A mixed method analysis of nurses with mental health problems who die by suicide

Pre-trained Language Models as Re-Annotators

Misspelling Semantics In Thai

Contact Info

Product

Resources

About