SKR-QA: Semantic ranking and knowledge revise for multi-choice question answering

Ren, Mucheng; Huang, Heyan

doi:10.1016/j.neucom.2021.06.076

Cited by 8 publications

(3 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fernández-Álvarez et al (2021) designed an importance ranking method on a complex knowledge graph to rank the class importance. In natural language, Ren et al (2021) evaluated the importance of knowledge from a semantic perspective. Zhong et al (2021) analyzed the correlation between candidate concepts through semantic similarity to calculate the semantic weight of concepts and extract important concepts.…”

Section: Knowledge-point Importance Discovery Methodsmentioning

confidence: 99%

Discovering Knowledge-Point Importance From the Learning-Evaluation Data

Guo

Wang

et al. 2022

International Journal of Distance Education Technologies

View full text Add to dashboard Cite

As students in online courses usually show differences in their cognitive levels and lack communication with teachers, it is difficult for teachers to grasp student perceptions of the importance of knowledge-points and to develop personalized teaching. Though recent studies have paid attention to this topic, existing methods fail to calculate the importance of every knowledge-point for each student. Moreover, some studies are based on expert analysis, are not data-driven, and hence are inapplicable to large-scale online scenarios. To address these issues, this article proposes a personal topic rank (PTR) as a solution, which links students and concepts to generate a personalized knowledge concept map. Then, the authors present a novel PTR method to calculate the importance of knowledge-points, wherein student mastery of knowledge-points, student understanding, and the knowledge-point itself are considered simultaneously. This article conducts extensive experiments on a real-world dataset to demonstrate that the method can achieve better results than baselines.

show abstract

Section: Knowledge-point Importance Discovery Methodsmentioning

confidence: 99%

Discovering Knowledge-Point Importance From the Learning-Evaluation Data

Guo

Wang

et al. 2022

International Journal of Distance Education Technologies

View full text Add to dashboard Cite

show abstract

“…Based on a similar idea, with post-hoc interpretation of intermediate results, the system by Ju et al [78] penalizes the attributions between the wrong answers and their supporting text snippets. Instead of only relying on the given context, Ren et al [79] tried to retrieve additional knowledge from an external corpus for context augmentation. Prioritizing speed, DeFormer [80] processes the context and the question independently in the lower layers of PLMs based on the observation that there is less variance in the lower layer representations of the text when jointly modeled with different questions.…”

Section: A An Overview Of Major Legal Nlp Tasksmentioning

confidence: 99%

On the Effectiveness of Pre-Trained Language Models for Legal Natural Language Processing: An Empirical Study

Song

Gao

et al. 2022

IEEE Access

View full text Add to dashboard Cite

We present the first comprehensive empirical evaluation of pre-trained language models (PLMs) for legal natural language processing (NLP) in order to examine their effectiveness in this domain.Our study covers eight representative and challenging legal datasets, ranging from 900 to 57K samples, across five NLP tasks: binary classification, multi-label classification, multiple choice question answering, summarization and information retrieval. We first run unsupervised, classical machine learning and/or non-PLM based deep learning methods on these datasets, and show that baseline systems' performance can be 4%∼35% lower than that of PLM-based methods. Next, we compare general-domain PLMs and those specifically pre-trained for the legal domain, and find that domain-specific PLMs demonstrate 1%∼5% higher performance than general-domain models, but only when the datasets are extremely close to the pretraining corpora. Finally, we evaluate six general-domain state-of-the-art systems, and show that they have limited generalizability to legal data, with performance gains from 0.1% to 1.2% over other PLM-based methods. Our experiments suggest that both general-domain and domain-specific PLM-based methods generally achieve better results than simpler methods on most tasks, with the exception of the retrieval task, where the best-performing baseline outperformed all PLM-based methods by at least 5%. Our findings can help legal NLP practitioners choose the appropriate methods for different tasks, and also shed light on potential future directions for legal NLP research.

show abstract

“…As deep learning and natural language processing technology rapidly advances, the question-answering system gradually transitions from early rule matching to retrieval matching [15]. The core idea is to extract the core words in natural language questions, search for the relevant answers in documents or web pages according to the core words and return the corresponding answers using the correlation sorting algorithm.…”

Section: Question Answering Systemmentioning

confidence: 99%

Enhancement of Question Answering System Accuracy via Transfer Learning and BERT

Duan

Du²,

Zhang³

et al. 2022

Applied Sciences

View full text Add to dashboard Cite

Entity linking and predicate matching are two core tasks in the Chinese Knowledge Base Question Answering (CKBQA). Compared with the English entity linking task, the Chinese entity linking is extremely complicated, making accurate Chinese entity linking difficult. Meanwhile, strengthening the correlation between entities and predicates is the key to the accuracy of the question answering system. Therefore, we put forward a Bidirectional Encoder Representation from Transformers and transfer learning Knowledge Base Question Answering (BAT-KBQA) framework, which is on the basis of feature-enhanced Bidirectional Encoder Representation from Transformers (BERT), and then perform a Named Entity Recognition (NER) task, which is appropriate for Chinese datasets using transfer learning and the Bidirectional Long Short-Term Memory-Conditional Random Field (BiLSTM-CRF) model. We utilize a BERT-CNN (Convolutional Neural Network) model for entity disambiguation of the problem and candidate entities; based on the set of entities and predicates, a BERT-Softmax model with answer entity predicate features is introduced for predicate matching. The answer ultimately chooses to integrate entities and predicates scores to determine the definitive answer. The experimental results indicate that the model, which is developed by us, considerably enhances the overall performance of the Knowledge Base Question Answering (KBQA) and it has the potential to be generalizable. The model also has better performance on the dataset supplied by the NLPCC-ICCPOL2016 KBQA task with a mean F1 score of 87.74% compared to BB-KBQA.

show abstract

SKR-QA: Semantic ranking and knowledge revise for multi-choice question answering

Cited by 8 publications

References 19 publications

Discovering Knowledge-Point Importance From the Learning-Evaluation Data

Discovering Knowledge-Point Importance From the Learning-Evaluation Data

On the Effectiveness of Pre-Trained Language Models for Legal Natural Language Processing: An Empirical Study

Enhancement of Question Answering System Accuracy via Transfer Learning and BERT

Contact Info

Product

Resources

About