ChatGPT, a language-learning model chatbot, has garnered considerable attention for its ability to respond to users’ questions. Using data from 14 countries and 186 institutions, we compare ChatGPT and student performance for 28,085 questions from accounting assessments and textbook test banks. As of January 2023, ChatGPT provides correct answers for 56.5 percent of questions and partially correct answers for an additional 9.4 percent of questions. When considering point values for questions, students significantly outperform ChatGPT with a 76.7 percent average on assessments compared to 47.5 percent for ChatGPT if no partial credit is awarded and 56.5 percent if partial credit is awarded. Still, ChatGPT performs better than the student average for 15.8 percent of assessments when we include partial credit. We provide evidence of how ChatGPT performs on different question types, accounting topics, class levels, open/closed assessments, and test bank questions. We also discuss implications for accounting education and research.
The ability to communicate effectively is critical in the accounting profession, yet research shows there is a gap between employer expectations and student abilities to adapt communication to various contexts. In this paper we introduce "Communication Roulette", a novel learning intervention that encourages students to practice their written communication skills by sensitizing them to the need to tailor their message to different audiences (i.e. shareholders, management, clients, etc.) using various communication formats (i.e. letter, email, presentation etc.) while simultaneously reinforcing their knowledge in important content areas. We provide implementation instructions and sample prompts, along with ideas to modify the intervention for a variety of classroom settings. Survey results indicate students find Communication Roulette increases both their confidence and ability to communicate effectively with different audiences and formats. We find these improvements in learning outcomes can largely be achieved through peer feedback alone, although we document incremental improvements in self-reported confidence with supplemental instructor feedback. Finally, pre- and post-assessments indicate an increase in student content knowledge following implementation of Communication Roulette.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.