DTC: Transfer learning for commonsense machine comprehension

Han, Weiguang; Peng, Min; Xie, Qianqian; Hu, Gang; Gao, Wang; Wang, Hua; Zhang, Yanchun; Liu, Zuopeng

doi:10.1016/j.neucom.2019.07.110

Cited by 9 publications

(5 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It should be noted that a relatively small number of dataset samples has a common basis in benchmarking due to expensive annotation or the need for expert competencies. Unlike datasets for machine-reading comprehension, such as MultiRC (Khashabi et al, 2018) and ReCoRD (Zhang et al, 2018), the GLUE-style datasets for learning choice of alternatives, logic, and causal relationships are often represented by a smaller number of manually collected and verified samples. They are by design sufficient for the human type of generalization but often pose a challenge for the tested LMs.…”

Section: Discussionmentioning

confidence: 99%

Ad astra or astray: Exploring linguistic knowledge of multilingual BERT through NLI task

et al. 2022

View full text Add to dashboard Cite

Recent research has reported that standard fine-tuning approaches can be unstable due to being prone to various sources of randomness, including but not limited to weight initialization, training data order, and hardware. Such brittleness can lead to different evaluation results, prediction confidences, and generalization inconsistency of the same models independently fine-tuned under the same experimental setup. Our paper explores this problem in natural language inference, a common task in benchmarking practices, and extends the ongoing research to the multilingual setting. We propose six novel textual entailment and broad-coverage diagnostic datasets for French, German, and Swedish. Our key findings are that the mBERT model demonstrates fine-tuning instability for categories that involve lexical semantics, logic, and predicate-argument structure and struggles to learn monotonicity, negation, numeracy, and symmetry. We also observe that using extra training data only in English can enhance the generalization performance and fine-tuning stability, which we attribute to the cross-lingual transfer capabilities. However, the ratio of particular features in the additional training data might rather hurt the performance for model instances. We are publicly releasing the datasets, hoping to foster the diagnostic investigation of language models (LMs) in a cross-lingual scenario, particularly in terms of benchmarking, which might promote a more holistic understanding of multilingualism in LMs and cross-lingual knowledge transfer.

show abstract

Section: Discussionmentioning

confidence: 99%

Ad astra or astray: Exploring linguistic knowledge of multilingual BERT through NLI task

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Reference [ 34 ] used learning networks to perform deep neural network learning through regularized approximations of decision trees. Reference [ 35 ] proposed an interpretable CNN for end-to-end learning, adding a priori constraints with filters to achieve automatic regression to a specific object (such as a bird's head, beak, and legs) after training and separating them in the top layer of the convolutional layer. Then, the representation of the neural network is refined into a decision tree structure [ 45 ], each decision mode hidden in the fully connected layer of CNN is encoded from coarse to fine, and the decision tree is used to approximate the final decision result.…”

Section: Methodsmentioning

confidence: 99%

Application of Machine Learning in Intelligent Medical Image Diagnosis and Construction of Intelligent Service Process

Gao

Lou

Wang

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

The introduction of digital technology in the healthcare industry is marked by ongoing difficulties with implementation and use. Slow progress has been made in unifying different healthcare systems, and much of the globe still lacks a fully integrated healthcare system. As a result, it is critical and advantageous for healthcare providers to comprehend the fundamental ideas of AI in order to design and deliver their own AI-powered technology. AI is commonly defined as the capacity of machines to mimic human cognitive functions. It can tackle jobs with equivalent or superior performance to humans by combining computer science, algorithms, machine learning, and data science. The healthcare system is a dynamic and evolving environment, and medical experts are constantly confronted with new issues, shifting duties, and frequent interruptions. Because of this variation, illness diagnosis frequently becomes a secondary concern for healthcare professionals. Furthermore, clinical interpretation of medical information is a cognitively demanding endeavor. This applies not just to seasoned experts, but also to individuals with varying or limited skills, such as young assistant doctors. In this paper, we proposed the comparative analysis of various state-of-the-art methods of deep learning for medical imaging diagnosis and evaluated various important characteristics. The methodology is to evaluate various important factors such as interpretability, visualization, semantic data, and quantification of logical relationships in medical data. Furthermore, the glaucoma diagnosis system is discussed in detail via qualitative and quantitative approaches. Finally, the applications and future prospects were also discussed.

show abstract

“…Based on the answer type, they differentiate cloze answer (the question is a sentence with a missing word which has to be inserted, e.g. ReCoRD [28]), selective or multiple choice (a number of options is given, and the correct one(s) should be selected, e.g. MultiRC [9]), boolean (a yes/no answer is expected, e.g.…”

Section: Related Workmentioning

confidence: 99%

“…Zhang et al [28] extracted their examples (more than 120 000 entries) from the CNN/Daily Mail 1 corpus to create the Reading Comprehension with Commonsense Reasoning (ReCoRD) dataset. These news articles were divided into multiple units: passage, cloze-style query (containing the masked entity) and the reference answer.…”

Section: Related Workmentioning

confidence: 99%

Building machine reading comprehension model from scratch

Yang¹,

Ligeti-Nagy²

2023

AMI

View full text Add to dashboard Cite

In this paper, we introduce a machine reading comprehension model and how we built this model from scratch. Reading comprehension is a crucial requisite for artificial intelligence applications, such as Question-Answering systems, chatbots, virtual assistants etc. Reading comprehension task requires the highest complexity of natural language processing methods. In recent years, the transformer neural architecture could achieve the ability to solve high complexity tasks. To make these applications available in Hungarian as well it is inevitable to design a Hungarian corpus of reading comprehension so that the pretrained models can be fine-tuned on this dataset.In our research, we have created the HuRC (Hungarian Reading Comprehension) corpus, which is the first dataset in Hungarian aiming to train, test and evaluate language models on a reading comprehension task. We built such a dataset based on the English ReCoRD corpus. This is a dataset of 120,000 examples consisting of news articles containing a passage and a close-style query, where a named entity is masked and the reference answer has to be found in a list.Using the evaluated dataset and transformers' question-answering library, we have built the first neural machine reading comprehension models in commonsense reasoning task for Hungarian.

show abstract

DTC: Transfer learning for commonsense machine comprehension

Cited by 9 publications

References 2 publications

Ad astra or astray: Exploring linguistic knowledge of multilingual BERT through NLI task

Ad astra or astray: Exploring linguistic knowledge of multilingual BERT through NLI task

Application of Machine Learning in Intelligent Medical Image Diagnosis and Construction of Intelligent Service Process

Building machine reading comprehension model from scratch

Contact Info

Product

Resources

About