How teachers conceptualise shared control with an AI co‐orchestration tool: A multiyear teacher‐centred design process

Abstract

Artificial intelligence (AI) can enhance teachers' capabilities by sharing control over different parts of learning activities. This is especially true for complex learning activities, such as dynamic learning transitions where students move between individual and collaborative learning in un‐planned ways, as the need arises. Yet, few initiatives have emerged considering how shared responsibility between teachers and AI can support learning and how teachers' voices might be included to inform design decisions. The goal of our article is twofold. First, we describe a secondary analysis of our co‐design process comprising six design methods to understand how teachers conceptualise sharing control with an AI co‐orchestration tool, called Pair‐Up. We worked with 76 middle school math teachers, each taking part in one to three methods, to create a co‐orchestration tool that supports dynamic combinations of individual and collaborative learning using two AI‐based tutoring systems. We leveraged qualitative content analysis to examine teachers' views about sharing control with Pair‐Up, and we describe high‐level insights about the human‐AI interaction, including control, trust, responsibility, efficiency, and accuracy. Secondly, we use our results as an example showcasing how human‐centred learning analytics can be applied to the design of human‐AI technologies and share reflections for human‐AI technology designers regarding the methods that might be fruitful to elicit teacher feedback and ideas. Our findings illustrate the design of a novel co‐orchestration tool to facilitate the transitions between individual and collaborative learning and highlight considerations and reflections for designers of similar systems. Practitioner notesWhat is already known about this topic: Artificial Intelligence (AI) can help teachers facilitate complex classroom activities, such as having students move between individual and collaborative learning in unplanned ways. Designers should use human‐centred design approaches to give teachers a voice in deciding what AI might do in the classroom and if or how they want to share control with it. What this paper adds: Presents teacher views about how they want to share control with AI to support students moving between individual and collaborative learning. Describes how we adapted six design methods to design AI features. Illustrates a complete, iterative process to create human‐AI interactions to support teachers as they facilitate students moving from individual to collaborative learning. Implications for practice: We share five implications for designers that teachers highlighted as necessary when designing AI‐features, including control, trust, responsibility, efficiency and accuracy. Our work also includes a reflection on our design process and implications for future design processes.

Human‐Centred Learning Analytics: 2019–24

“…In 'How teachers conceptualise shared control with an AI co-orchestration tool: a multiyear teacher-centred design process', Lawrence et al (2023) report on a secondary study regarding a long-term design project on shared control between teachers and an AI co-orchestration tool. The authors address the wider field of Human-Centered Artificial Intelligence (HCAI) drawing on and discussing multiple known design methods used in HCI and HCLA.…”

“…They also flag how various instructional strategies at different social levels ultimately shape the design considerations of the learning analytics interfaces. Lawrence et al (2023) compile and analyse the results of six design methods, namely classroom probe, storyboard survey, low-fidelity and mid-fidelity prototyping, user testing and classroom study. They suggest that ways should be found to adapt methods to be more grounded in teacher classroom norms and student learning needs.…”

“…This was based on a 20-year-long RPP, while the specific co-design process took place over three academic years. Lawrence et al (2023) reported a secondary analysis implementing six HCD methods for over 2.5 years. Thus, they were able to 'expand on our past analyses to reflect specifically on how teachers conceptualise control across our entire design process to gain insights about how we might design for shared control in the future.…”

“…Wiley et al (2023) also focus on K-12 education, and suggest that 'Designing in the K-12 context presents unique challenges that warrant the use of HCLA design approaches'. Lawrence et al (2023) addressed seventh-grade classes, and their extensive design and implementation studies showed that the HCAI processes were especially useful for the K-12 context. Hutchins and Biswas (2023) co-designed a dashboard with middle school teachers, focusing on problem-based learning pedagogical approaches.…”

“…and Are we seeing richer definitions of 'human-centredness'? Apart from Lawrence et al (2023) who discuss human-AI teaming, none of the papers addressed these, but they are worth posing once again, on at least two grounds.…”