Utilizing Bidirectional Encoder Representations from Transformers for Answer Selection

Laskar, Md Tahmid Rahman; Hoque, Enamul; Huang, Jimmy Xiangji

doi:10.1007/978-3-030-63591-6_63

Cited by 13 publications

(15 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Earlier work primarily relies on feature engineering and linguistic information [60]. However, the advancement of deep learning introduces powerful models [61], [62] that outperform traditional methods without the need of manual efforts or feature engineering.…”

Section: Answer Selectionmentioning

confidence: 99%

“…Once the extended dataset is generated, we can apply any answer selection method to the dataset. There are a number of studies in the literature on this topic, including COALA [61], CETE [62], MTQA [79], and many more, among which CETE is considered state-of-the-art in the answer selection task by the ACL community 4 . CETE implements a transformer-based encoder (e.g., BERT) to encode the question and answer pair into a single vector and calculates the probability that a pair of question/answer should match or not 5 .…”

Section: Experimental Settingsmentioning

confidence: 99%

See 1 more Smart Citation

AgriBERT: Knowledge-Infused Agricultural Language Models for Matching Food and Nutrition

Tabar

Jung

Yadav

et al. 2022

Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

Eviction of tenants has reached a crisis level in the U.S. and its consequences pose significant challenges to society. To tackle this eviction crisis, policymakers have been allocating financial resources but a more efficient resource allocation would need an accurate forecast of the number of tenants at-risk of evictions ahead of time. To help enhance the existing eviction prevention/diversion programs, in this work, we propose a multi-view deep neural network model, named as MARTIAN, that forecasts the number of tenants at-risk of getting formally evicted (at the census tract level) n months into the future. Then, we evaluate MARTIAN’s predictive performance under various conditions using real-world eviction cases filed across Dallas County, TX. The results of empirical evaluation show that MARTIAN outperforms an extensive set of baseline models in terms of predictive performance. Additionally, MARTIAN’s superior predictive performance is generalizable to unseen census tracts, for which no labeled data is available in the training set. This research has been done in collaboration with Child Poverty Action Lab (CPAL), which is a pioneering non-governmental organization (NGO) working for tackling poverty-related issues across Dallas County, TX. The usability of MARTIAN is under review by subject matter experts. We release our codebase at https://github.com/maryam-tabar/MARTIAN.

show abstract

Section: Answer Selectionmentioning

confidence: 99%

Section: Experimental Settingsmentioning

confidence: 99%

AgriBERT: Knowledge-Infused Agricultural Language Models for Matching Food and Nutrition

Tabar

Jung

Yadav

et al. 2022

Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

show abstract

“…With the widespread application of pre-trained language model, the accuracy of answer selection task can also be improved by pre-trained method. Laskar et al [35] used pre-trained language model BERT for answer selection tasks, which can effectively leverage the context of each word in a sentence and improve performance of the model.…”

Section: Answer Selectionmentioning

confidence: 99%

Joint Learning With BERT-GCN and Multi-Attention for Event Text Classification and Event Assignment

She¹,

Chen²,

Chen³

2022

IEEE Access

View full text Add to dashboard Cite

Government hotline is closely related to people's lives and plays an important role in solving social problems and maintaining social stability in China. However, the event text of the hotline is inconsistent in length and unclear in elements, so it is a challenge for the operator to manually complete the assignment tasks of hotline event. To address these problems, we propose a joint learning method for event text classification and event assignment for Chinese government hotline. Firstly, graph convolution network (GCN) and BERT are used to process the event text respectively to obtain the corresponding representation vector. Then, the obtained two representation vectors are fused by the dynamic fusion gate to get fusion vector and classified the fusion vector through the text classification. Secondly, we use multi-attention mechanism to process the GCN result vector, BERT result vector and the "sanding" vector to obtain attentive "eventsanding" representation vector and calculate the corresponding department probability distribution. Finally, the historical prior knowledge based reorder model is used to sort the results of the "event-sanding" matching module and output the optimal assignment department of government hotline event. Experimental results show that our method can achieve better performance compared with several baseline approaches. The ablation experiments also demonstrate the validity of each proposed module in our model.

show abstract

“…Then, we fine-tune the pre-trained transformer-based summarization model on each individual document using the weak reference summaries. In this way, we generate the summary of each individual document and then select the most relevant sentences as the final summary using a transformer-based answer selection model (Laskar, Huang, and Hoque 2020;Laskar, Hoque, and Huang 2020b). In another approach, instead of training our model on each document, we again utilize a transformer-based answer selection model and construct a filtered input document via selecting the sentences (up to n tokens) in the document set that are most relevant to the query.…”

Section: Extending Preqfas For Long Sequences In the Md-qfas Taskmentioning

confidence: 99%

Domain Adaptation with Pre-trained Transformers for Query-Focused Abstractive Text Summarization

Laskar

Hoque

Huang

2022

Computational Linguistics

Self Cite

View full text Add to dashboard Cite

The Query Focused Text Summarization (QFTS) task aims at building systems that generate the summary of the text document(s) based on the given query. A key challenge in addressing this task is the lack of large labeled data for training the summarization model. In this paper, we address this challenge by exploring a series of domain adaptation techniques. Given the recent success of pre-trained transformer models in a wide range of natural language processing tasks, we utilize such models to generate abstractive summaries for the QFTS task for both single-document and multi-document scenarios. For domain adaptation, we apply a variety of techniques using pre-trained transformer-based summarization models including transfer learning, weakly supervised learning, and distant supervision. Extensive experiments on six datasets show that our proposed approach is very effective in generating abstractive summaries for the QFTS task while setting a new state-of-the-art result in several datasets across a set of automatic and human evaluation metrics.

show abstract

Utilizing Bidirectional Encoder Representations from Transformers for Answer Selection

Cited by 13 publications

References 24 publications

AgriBERT: Knowledge-Infused Agricultural Language Models for Matching Food and Nutrition

AgriBERT: Knowledge-Infused Agricultural Language Models for Matching Food and Nutrition

Joint Learning With BERT-GCN and Multi-Attention for Event Text Classification and Event Assignment

Domain Adaptation with Pre-trained Transformers for Query-Focused Abstractive Text Summarization

Contact Info

Product

Resources

About