Language Models are Few-shot Multilingual Learners

Winata, Genta Indra; Madotto, Andrea; Lin, Zhaojiang; Liu, Rosanne; Yosinski, Jason; Fung, Pascale

doi:10.18653/v1/2021.mrl-1.1

Cited by 692 publications

(1,070 citation statements)

References 16 publications

Supporting

Mentioning

658

Contrasting

Unclassified

Order By: Relevance

“…For example, BERT [2] is a pre-trained transformer-based encoder model that can be fine-tuned on various NLP tasks, such as sentence classification, question answering and named entity recognition. In fact, the so-called few-shot learning capability of large language models to be efficiently adapted to down-stream tasks or even other seemingly unrelated tasks (e.g., as in transfer learning) has been empirically observed and studied for various natural-language tasks [6], e.g., more recently in the context of generating synthetic and yet realistic heterogeneous tabular data [7].…”

Section: Introductionmentioning

confidence: 99%

ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education

Kasneci¹,

Seßler²,

Küchemann³

et al. 2023

Preprint

View full text Add to dashboard Cite

Large language models represent a significant advancement in the field of AI. The underlying technology is key to further innovations and, despite critical views and even bans within communities and regions, large language models are here to stay. This position paper presents the potential benefits and challenges of educational applications of large language models, from student and teacher perspectives. We briefly discuss the current state of large language models and their applications. We then highlight how these models can be used to create educational content, improve student engagement and interaction, and personalize learning experiences. With regard to challenges, we argue that large language models in education require teachers and learners to develop sets of competencies and literacies necessary to both understand the technology as well as their limitations and unexpected brittleness of such systems. In addition, a clear strategy within educational systems and a clear pedagogical approach with a strong focus on critical thinking and strategies for fact checking are required to integrate and take full advantage of large language models in learning settings and teaching curricula. Other challenges such as the potential bias in the output, the need for continuous human oversight, and the potential for misuse are not unique to the application of AI in education. But we believe that, if handled sensibly, these challenges can offer insights and opportunities in education scenarios to acquaint students early on with potential societal biases, criticalities, and risks of AI applications. We conclude with recommendations for how to address these challenges and ensure that such models are used in a responsible and ethical manner in education.

show abstract

Section: Introductionmentioning

confidence: 99%

ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education

Kasneci¹,

Seßler²,

Küchemann³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Deep learning (DL) models have attained excellent results in a diversity of problems, in classification large-scale images [1], processing of natural language [2], and segmentation of the medical images [3]. However, standard approaches are discovered to produce more predictions than necessary, which means they are not calibrated correctly [4].…”

Section: Introductionmentioning

confidence: 99%

Bayesian deep learning methods applied to diabetic retinopathy disease: a review

Ismail

Hassan²

2023

IJEECS

View full text Add to dashboard Cite

Diabetic retinopathy (DR) is a complication of diabetes that cause retinal damage; therefore, it is a leading cause of blindness. However, early detection of this disease can dramatically reduce the risk of vision loss. The main problem of early DR detection is that the manual diagnosis by ophthalmology is time-consuming, expensive, and prone to misdiagnosis. Deep learning (DL) models have aided in the early diagnosis of DR, and DL is now frequently utilized in DR detection and classification. The main issues with classical DL models is that they are incapable to quantify the uncertainty in the models, thus they are prone to make wrong decisions in complex cases. However, Bayesian deep learning (BDL) models have recently evolved as unified probabilistic framework to integrate DL and Bayesian models to provides an accurate framework to identify all sources of uncertainty in the model. This paper introduces BDL and most recent research that used BDL approaches to treat diabetic retinopathy are reviewed and discussed. A thorough comparison of the existing Bayesian approaches in this topic is also presented. In addition, available datasets for the fundus retina, which is often employed in DR, are provided and reviewed.

show abstract

“…There is a problem with reliability and interpretability. 5 Therefore, we should not use the outcomes of such language models in contexts where an incorrect output is ethically questionable. Since outputs can be subtly flawed or untrue, one has to remain generally skeptical.…”

Section: Introductionmentioning

confidence: 99%

How Far Can We Get in Creating a Digital Replica of a Philosopher?

Strasser

Crosby

Schwitzgebel

2023

Social Robots in Social Institutions

View full text Add to dashboard Cite

Can we build machines with which we can have interesting conversations? Observing the new optimism of AI regarding deep learning and new language models, we set ourselves an ambitious goal: We want to find out how far we can get in creating a digital replica of a philosopher. This project has two aims; one more technical, investigating of how the best model can be built. The other one, more philosophical, exploring the limitations and risks which accompany the creation of digital replicas. In cooperation with Daniel Dennett, we took his complete works to fine-tune a GPT-3. In this paper we share our first results with some piloting models and discuss legal and ethical questions about creating and interacting with digital replicas.

show abstract

Language Models are Few-shot Multilingual Learners

Cited by 692 publications

References 16 publications

ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education

ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education

Bayesian deep learning methods applied to diabetic retinopathy disease: a review

How Far Can We Get in Creating a Digital Replica of a Philosopher?

Contact Info

Product

Resources

About