War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education

doi:10.37074/jalt.2023.6.1.23

Cited by 45 publications

(10 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However the latest foundation model GPT-4 consistently outperforms GPT-3.5 and often scores nearly 50% on exams, this is shown in figure 5. Given these results, and as highlighted at the end of [6], the current potential threat of AI in non-invigilated online exams at university level seems to be relatively contained. In fact, it would be prudent to warn students that AI performance at GCSE and A Level may not transfer to university assessments.…”

Section: Discussionmentioning

confidence: 84%

“…Following the prominence of ChatGPT, tech behemoths such as Meta, Google, and Baidu have launched their own AI-driven chatbots: LLama, Gemini [5], and Ernie. While these models excel in various tasks, preliminary research indicates that they may not consistently meet the rigorous academic standards of university settings [6], with GPT-4, the latest iteration of the GPT series, outperforming its counterparts. 1 Available at https://openai.com/blog/teaching-with-ai.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The impact of AI in physics education: a comprehensive review from GCSE to university levels

Yeadon,

Hardy

2024

Phys. Educ.

View full text Add to dashboard Cite

With the rapid evolution of artificial intelligence (AI), its potential implications for higher education have become a focal point of interest. This study delves into the capabilities of AI in physics education and offers actionable AI policy recommendations. Using openAI’s flagship gpt-3.5-turbo large language model (LLM), we assessed its ability to answer 1337 physics exam questions spanning general certificate of secondary education (GCSE), A-Level, and introductory university curricula. We employed various AI prompting techniques: Zero Shot, in context learning, and confirmatory checking, which merges chain of thought reasoning with reflection. The proficiency of gpt-3.5-turbo varied across academic levels: it scored an average of 83.4% on GCSE, 63.8% on A-Level, and 37.4% on university-level questions, with an overall average of 59.9% using the most effective prompting technique. In a separate test, the LLM’s accuracy on 5000 mathematical operations was found to be 45.2%. When evaluated as a marking tool, the LLM’s concordance with human markers averaged at 50.8%, with notable inaccuracies in marking straightforward questions, like multiple-choice. Given these results, our recommendations underscore caution: while current LLMs can consistently perform well on physics questions at earlier educational stages, their efficacy diminishes with advanced content and complex calculations. LLM outputs often showcase novel methods not in the syllabus, excessive verbosity, and miscalculations in basic arithmetic. This suggests that at university, there’s no substantial threat from LLMs for non-invigilated physics questions. However, given the LLMs’ considerable proficiency in writing physics essays and coding abilities, non-invigilated examinations of these skills in physics are highly vulnerable to automated completion by LLMs. This vulnerability also extends to pysics questions pitched at lower academic levels. It is thus recommended that educators be transparent about LLM capabilities with their students, while emphasizing caution against overreliance on their output due to its tendency to sound plausible but be incorrect.

show abstract

Section: Discussionmentioning

confidence: 84%

Section: Introductionmentioning

confidence: 99%

The impact of AI in physics education: a comprehensive review from GCSE to university levels

Yeadon,

Hardy

2024

Phys. Educ.

View full text Add to dashboard Cite

show abstract

“…ChatGPT, Bing Chat, and other similar techniques seem to be promising in solving such problems. In-depth model knowledge should be merged with these artificial intelligence (AI) tools to support better usage of geoscientific models (Chen et al, 2023;OpenAI, 2023;Rudolph et al, 2023). The GPT-like tools can also be promising in automatically generating knowledge contents for the model framework, for example, using ChatGPT reading an article introducing a model and then automatically offering the basic descriptions and a conceptual diagram explaining the model.…”

Section: Knowledge Applicationmentioning

confidence: 99%

Construction of an open knowledge framework for geoscientific models

Xu,

Yue,

Chen

et al. 2024

Transactions in GIS

View full text Add to dashboard Cite

Geoscientific models have rapidly developed in recent decades as effective tools to understand the past, perceive the present, and predict the future. However, with the increasing number of models available, discovering suitable ones and applying them properly in problem‐solving situations has become more challenging. Existing materials describing geoscientific models (e.g., articles, manuals, handbooks, metadata documents, and web pages) are scattered and varied in structure and content. To help model users from different disciplinary backgrounds find, access, implement, and reuse models more conveniently, we propose an open knowledge framework for geoscientific models. The knowledge framework includes three levels: a resource level for indicating where to find a model, a connection level for indicating what materials are related to a model, and an application level for indicating how the model is applied. Through such a three‐level framework, model users can collaboratively provide descriptive information for a model, link different materials to a model (e.g., data, references, and tools), and contribute experiences regarding model application in practical cases (as reusable solutions). Thus, a web‐based community can be formed to facilitate the better use of geoscientific models. This article introduces the Open Geographic Modeling and Simulation System (OpenGMS) as the implementation of this open knowledge framework. Case studies are given to showcase the effectiveness and capability of the proposed framework.

show abstract

“…The impact of GenAI tools on teaching, learning and assessment practice in HE is a hotly debated topic (Rudolph et al, 2023a(Rudolph et al, , 2023b. The use of authentic assessment is widespread in UK universities, and the adoption of it has been praised over traditional forms of assessment as it appears to have reduced the negative effect of class size on student attainment (Richardson, 2015).…”

Section: Introductionmentioning

confidence: 99%

Higher education assessment practice in the era of generative AI tools

2024

JALT

View full text Add to dashboard Cite

The higher education (HE) sector benefits every nation's economy and society at large. However, their contributions are challenged by advanced technologies like generative artificial intelligence (GenAI) tools. In this paper, we provide a comprehensive assessment of GenAI tools towards assessment and pedagogic practice and, subsequently, discuss the potential impacts. This study experimented using three assessment instruments from data science, data analytics, and construction management disciplines. Our findings are two-fold: first, the findings revealed that GenAI tools exhibit subject knowledge, problem-solving, analytical, critical thinking, and presentation skills and thus can limit learning when used unethically. Secondly, the design of the assessment of certain disciplines revealed the limitations of the GenAI tools. Based on our findings, we made recommendations on how AI tools can be utilised for teaching and learning in HE.

show abstract

War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education

Abstract: Jo u r n a l o f A p p l ie d Lea r n in g & T e a c h in g JALT http://journals.sfu.ca/jalt/index.php/jalt/index

Cited by 45 publications

References 56 publications

The impact of AI in physics education: a comprehensive review from GCSE to university levels

The impact of AI in physics education: a comprehensive review from GCSE to university levels

Construction of an open knowledge framework for geoscientific models

Higher education assessment practice in the era of generative AI tools

Contact Info

Product

Resources

About