ChatGPT effects on cognitive skills of undergraduate students: Receiving instant responses from AI-based conversational large language models (LLMs)

Essel, Harry Barton; Vlachopoulos, Dimitrios; Essuman, Albert Benjamin; Amankwa, John Opuni

doi:10.1016/j.caeai.2023.100198

Cited by 31 publications

(15 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It can act as a beneficial guide, helping students start a project and igniting their analytical thinking. This approach is particularly aligned with the cognitive skill enhancement addressed in Essel et al (2024) which noted that incorporating ChatGPT for in-class tasks effectively improved students' critical thinking skills.…”

Section: Influence On Othersmentioning

confidence: 99%

Navigating the AI Revolution: Implications for Business Education and Pedagogy

2024

JCT

View full text Add to dashboard Cite

Generative artificial intelligence (AI) is rapidly emerging as a transformative force across various sectors. As education shifts towards an AI-focused future, adapting teaching methodologies and evaluation strategies to this technological evolution becomes more and more important. This paper delves into the profound implications of generative AI on business education, critically analyzing its influence on broad program learning outcomes as well as on specific assessment tasks, ranging from quizzes to work-integrated learning projects. By examining and assessing responses from ChatGPT, we evaluate their structural coherence and the potential to enhance or replace key skills evaluated in students. Our findings primarily indicate that for formulaic quizzes and short essay questions, GPT-4 often delivers accurate solutions. In the context of research reports and reflection journals, GPT-4 serves more as a guide and scaffolding tool. As AI continues to advance, it becomes increasingly crucial for business educators to re-examine their learning objective frameworks. This includes utilizing the technology’s potential while navigating its complexities, ensuring that education remains effective, relevant, and in step with the pace of innovation.

show abstract

Section: Influence On Othersmentioning

confidence: 99%

Navigating the AI Revolution: Implications for Business Education and Pedagogy

2024

JCT

View full text Add to dashboard Cite

show abstract

“…This adaptation involves the need to emphasize educational aspects such as self-reflection, critical thinking, problem-solving, and independent learning [33]. Consequently, the educational systems can benefit from AI as a tool to complement, rather than replace, human intellect and creativity [34].…”

Section: Introductionmentioning

confidence: 99%

Human versus Artificial Intelligence: ChatGPT-4 Outperforming Bing, Bard, ChatGPT-3.5, and Humans in Clinical Chemistry Multiple-Choice Questions

Sallam,

Al-Salahat,

Eid

et al. 2024

Preprint

View full text Add to dashboard Cite

The advances in large language models (LLMs) are evolving rapidly. Artificial intelligence (AI) chatbots based on LLMs excel in language understanding and generation, with potential utility to transform healthcare education and practice. However, it is important to assess the performance of such AI models in various topics to highlight its strengths and possible limitations. Therefore, this study aimed to evaluate the performance of ChatGPT (GPT-3.5 and GPT-4), Bing, and Bard compared to human students at a postgraduate master’s (MSc) level in Medical Laboratory Sciences. The study design was based on the METRICS checklist for the design and reporting of AI-based studies in healthcare. The study utilized a dataset of 60 Clinical Chemistry multiple-choice questions (MCQs) initially conceived for assessment of 20 MSc students. The revised Bloom’s taxonomy was used as the framework for classifying the MCQs into four cognitive categories: Remember, Understand, Analyze, and Apply. A modified version of the CLEAR tool was used for assessment of the quality of AI-generated content, with Cohen’s κ for inter-rater agreement. Compared to the mean students’ score which was 40/60 (66.8%), GPT-4 scored 54/60 (90.0%), followed by Bing (46/60, 76.7%), GPT-3.5 (44/60, 73.3%), and Bard (40/60, 66.7%). Statistically significant better performance was noted in lower cognitive domains (Remember and Understand) in GPT-3.5, GPT-4, and Bard. The CLEAR scores indicated that ChatGPT-4 performance was “Excellent” compared to “Above average” performance of ChatGPT-3.5, Bing, and Bard. The findings indicated that ChatGPT-4 excelled in the Clinical Chemistry exam, while ChatGPT-3.5, Bing, and Bard were above-average. Given that the MCQs were directed to postgraduate students with a high degree of specialization, the performance of these AI chatbots was remarkable. Due to the risks of academic dishonesty and possible dependence on these AI models, the appropriateness of MCQs as an assessment tool in higher education should be re-evaluated.

show abstract

“…This adaptation involves the need to emphasize educational aspects such as self-re ection, critical thinking, problem-solving, and independent learning [33]. Consequently, the educational systems can bene t from AI as a tool to complement, rather than replace, human intellect and creativity [34].…”

Section: Introductionmentioning

confidence: 99%

Human versus Artificial Intelligence: ChatGPT-4 Outperforming Bing, Bard, ChatGPT-3.5, and Humans in Clinical Chemistry Multiple-Choice Questions

Sallam,

Al-Salahat,

Eid

et al. 2024

Preprint

View full text Add to dashboard Cite

Background The advances in large language models (LLMs) are evolving rapidly. Artificial intelligence (AI) chatbots based on LLMs excel in language understanding and generation, with potential utility to transform healthcare education and practice. However, it is important to assess the performance of such AI models in various topics to highlight its strengths and possible limitations. Therefore, this study aimed to evaluate the performance of ChatGPT (GPT-3.5 and GPT-4), Bing, and Bard compared to human students at a postgraduate master’s (MSc) level in Medical Laboratory Sciences. Methods The study design was based on the METRICS checklist for the design and reporting of AI-based studies in healthcare. The study utilized a dataset of 60 Clinical Chemistry multiple-choice questions (MCQs) initially conceived for assessment of 20 MSc students. The revised Bloom’s taxonomy was used as the framework for classifying the MCQs into four cognitive categories: Remember, Understand, Analyze, and Apply. A modified version of the CLEAR tool was used for assessment of the quality of AI-generated content, with Cohen’s κ for inter-rater agreement. Results Compared to the mean students’ score which was 40/60 (66.8%), GPT-4 scored 54/60 (90.0%), followed by Bing (46/60, 76.7%), GPT-3.5 (44/60, 73.3%), and Bard (40/60, 66.7%). Statistically significant better performance was noted in lower cognitive domains (Remember and Understand) in GPT-3.5, GPT-4, and Bard. The CLEAR scores indicated that ChatGPT-4 performance was “Excellent” compared to “Above average” performance of ChatGPT-3.5, Bing, and Bard. Conclusions The findings indicated that ChatGPT-4 excelled in the Clinical Chemistry exam, while ChatGPT-3.5, Bing, and Bard were above-average. Given that the MCQs were directed to postgraduate students with a high degree of specialization, the performance of these AI chatbots was remarkable. Due to the risks of academic dishonesty and possible dependence on these AI models, the appropriateness of MCQs as an assessment tool in higher education should be re-evaluated.

show abstract

ChatGPT effects on cognitive skills of undergraduate students: Receiving instant responses from AI-based conversational large language models (LLMs)

Cited by 31 publications

References 38 publications

Navigating the AI Revolution: Implications for Business Education and Pedagogy

Navigating the AI Revolution: Implications for Business Education and Pedagogy

Human versus Artificial Intelligence: ChatGPT-4 Outperforming Bing, Bard, ChatGPT-3.5, and Humans in Clinical Chemistry Multiple-Choice Questions

Human versus Artificial Intelligence: ChatGPT-4 Outperforming Bing, Bard, ChatGPT-3.5, and Humans in Clinical Chemistry Multiple-Choice Questions

Contact Info

Product

Resources

About