AUTOHOME-ORCA at SemEval-2019 Task 8: Application of BERT for Fact-Checking in Community Forums

Lv, Zhengwei; Liu, Duoxing; Sun, Haifeng; Liang, Xiao; Leí, Tao; Shi, Zhizhong; Zhu, Feng; Yang, Ling

doi:10.18653/v1/s19-2150

Cited by 7 publications

(8 citation statements)

References 18 publications

(17 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Results for the pretraining experiments (shown in Table 7) show significant improvement of the pretrained model over the models trained only on our corpus. This is similar to findings by Lv et al (2019). However, the improvement is disproportionally larger in the stance prediction task (76.7 vs. 37.8 F 1 ) and the large gains do not carry over to the claim verification task (64.3 vs. 53.1 F 1 ).…”

Section: Resultssupporting

confidence: 86%

“…Pretrained Transformer: Pretraining and transfer learning (Devlin et al, 2018a; has recently gained attention as a popular approach to acquiring universal linguistic features useful in many downstream NLP tasks and was shown to be successful in improving on the state of the art in many downstream NLP tasks with minimal fine-tuning. Lv et al (2019) have successfully explored BERT for the task of fake news detection in English and proposed an extension that improves on fine-tuned BERT. In addition to the aforementioned supervised methods, we evaluate BERT (Devlin et al, 2018a) on both tasks in our corpus.…”

Section: Methodsmentioning

confidence: 99%

“…B-Contradict the original title: Looks similar to the original title but has contradicting meaning (both cannot be true in the same context) by reversing meaning without negating main verb, using antonym of main verb with rephrasing, changing key information using world knowledge such as locations, counts and dates. 2017; Alzanin and Azmi, 2019) including deep learning techniques (Hanselowski et al, 2017;Baly et al, 2018b;Popat et al, 2018;Chawla et al, 2019;Helwe et al, 2019;Lv et al, 2019).…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Proceedings of the Third Workshop on Fact Extraction and VERification (FEVER)

2020

View full text Add to dashboard Cite

Automatic fact checking is an important task motivated by the need for detecting and preventing the spread of misinformation across the web. The recently released FEVER challenge provides a benchmark task that assesses systems' capability for both the retrieval of required evidence and the identification of auforms the typical multi-task learning approach, and (2) gets comparable results to top performing systems with a much simpler training setup and less training computation (in terms of the amount of data consumed and the number of model parameters), facilitating future works on the automatic fact checking task and its practical usage. * Equal contribution Pip. Mul. Mix. Cmp.

show abstract

Section: Resultssupporting

confidence: 86%

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Proceedings of the Third Workshop on Fact Extraction and VERification (FEVER)

2020

View full text Add to dashboard Cite

show abstract

“…Pretrained Transformer: Pretraining and transfer learning (Devlin et al, 2018a;Peters et al, 2018;Radford et al, 2019) has recently gained attention as a popular approach to acquiring universal linguistic features useful in many downstream NLP tasks and was shown to be successful in improving on the state of the art in many downstream NLP tasks with minimal fine-tuning. Lv et al (2019) have successfully explored BERT for the task of fake news detection in English and proposed an extension that improves on fine-tuned BERT. In addition to the aforementioned supervised methods, we evaluate BERT (Devlin et al, 2018a) on both tasks in our corpus.…”

Section: Methodsmentioning

confidence: 99%

“…B-Contradict the original title: Looks similar to the original title but has contradicting meaning (both cannot be true in the same context) by reversing meaning without negating main verb, using antonym of main verb with rephrasing, changing key information using world knowledge such as locations, counts and dates. 2017; Wang et al, 2018;Alzanin and Azmi, 2019) including deep learning techniques (Hanselowski et al, 2017;Baly et al, 2018b;Popat et al, 2018;Chawla et al, 2019;Helwe et al, 2019;Lv et al, 2019).…”

Section: Related Workmentioning

confidence: 99%

Stance Prediction and Claim Verification: An Arabic Perspective

Khouja¹

2020

Proceedings of the Third Workshop on Fact Extraction and VERification (FEVER)

View full text Add to dashboard Cite

This work explores the application of textual entailment in news claim verification and stance prediction using a new corpus in Arabic. The publicly available corpus comes in two perspectives: a version consisting of 4,547 true and false claims and a version consisting of 3,786 pairs (claim, evidence). We describe the methodology for creating the corpus and the annotation process. Using the introduced corpus, we also develop two machine learning baselines for two proposed tasks: claim verification and stance prediction. Our best model utilizes pretraining (BERT) and achieves 76.7 F1 on the stance prediction task and 64.3 F1 on the claim verification task. Our preliminary experiments shed some light on the limits of automatic claim verification that relies on claims text only. Results hint that while the linguistic features and world knowledge learned during pretraining are useful for stance prediction, such learned representations from pretraining are insufficient for verifying claims without access to context or evidence.

show abstract

Attention Mechanism with BERT for Content Annotation and Categorization of Pregnancy-Related Questions on a Community Q&A Site

Luo

Ding

Tang

et al. 2020

2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

View full text Add to dashboard Cite

In recent years, the social web has been increasingly used for health information seeking, sharing, and subsequent health-related research. Women often use the Internet or social networking sites to seek information related to pregnancy in different stages. They may ask questions about birth control, trying to conceive, labor, or taking care of a newborn or baby. Classifying different types of questions about pregnancy information (e.g., before, during, and after pregnancy) can inform the design of social media and professional websites for pregnancy education and support. This research aims to investigate the attention mechanism built-in or added on top of the BERT model in classifying and annotating the pregnancy-related questions posted on a community Q&A site. We evaluated two BERT-based models and compared them against the traditional machine learning models for question classification. Most importantly, we investigated two attention mechanisms: the built-in self-attention mechanism of BERT and the additional attention layer on top of BERT for relevant term annotation. The classification performance showed that the BERTbased models worked better than the traditional models, and BERT with an additional attention layer can achieve higher overall precision than the basic BERT model. The results also showed that both attention mechanisms work differently on annotating relevant content, and they could serve as feature selection methods for text mining in general.

show abstract

AUTOHOME-ORCA at SemEval-2019 Task 8: Application of BERT for Fact-Checking in Community Forums

Cited by 7 publications

References 18 publications

Proceedings of the Third Workshop on Fact Extraction and VERification (FEVER)

Proceedings of the Third Workshop on Fact Extraction and VERification (FEVER)

Stance Prediction and Claim Verification: An Arabic Perspective

Attention Mechanism with BERT for Content Annotation and Categorization of Pregnancy-Related Questions on a Community Q&A Site

Contact Info

Product

Resources

About