Robosourcing Educational Resources -- Leveraging Large Language Models for Learnersourcing

Abstract

In this article, we introduce and evaluate the concept of robosourcing for creating educational content. Robosourcing lies in the intersection of crowdsourcing and large language models, where requests to large language models replace some of the work traditionally performed by the crowd. Robosourcing includes a human-in-the-loop to provide priming (input) as well as to evaluate and potentially adjust the generated artefacts; these evaluations could also be used to improve the large language models. We explore the feasibility of robosourcing in the context of education by conducting an evaluation of robosourced programming exercises, generated using OpenAI Codex. Our results suggest that robosourcing could significantly reduce human effort in creating diverse educational content while maintaining quality similar to human-created content. Thus, we argue that robosourcing has the potential to alleviate known issues around learner motivation and content quality that have been shown to limit the benefits of learnersourcing in practice.

Use of AI-driven Code Generation Models in Teaching and Learning Programming: a Systematic Literature Review

“…Three papers [23,5,10] extensively explain and evaluate an educational tool that uses LLM-based code generation models. This subsection explores the Robosourcing model [5],…”

“…Robosourcing [5] is a model for generating practice questions in a scalable manner as a learnersourcing activity. Learnersourcing involves learners collectively creating content for future learners while enhancing their own learning experience [21].…”

Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Language

“…It became generally available to all developers in June of 2022, and it is currently free for students and teachers 3 . The impact on educational practice of such technologies is unknown, with arguments on both sides-highlighting concerns of over-reliance by novices [6], and suggesting that the ability to synthesize code automatically could play a revolutionary role in teaching [9,13].…”

Transformed by Transformers: Navigating the AI Coding Revolution for Computing Education: An ITiCSE Working Group Conducted by Humans

“…The emergence of large language models could also bring benefits, however. These models can be used to scaffold instructors in creating educational resources such as novel, personalized programming exercises [2,10], code explanations that could help support students when they are working on exercises [6,8], and enhanced programming error messages that might be easier to understand for novice programmers [7].…”