Computationally Identifying Funneling and Focusing Questions in Classroom Discourse

Abstract

Responsive teaching is a highly effective strategy that promotes student learning. In math classrooms, teachers might funnel students towards a normative answer or focus students to reflect on their own thinking, deepening their understanding of math concepts. When teachers focus, they treat students' contributions as resources for collective sensemaking, and thereby significantly improve students' achievement and confidence in mathematics. We propose the task of computationally detecting funneling and focusing questions in classroom discourse. We do so by creating and releasing an annotated dataset of 2,348 teacher utterances labeled for funneling and focusing questions, or neither. We introduce supervised and unsupervised approaches to differentiating these questions. Our best model, a supervised RoBERTa model fine-tuned on our dataset, has a strong linear correlation of .76 with human expert labels and with positive educational outcomes, including math instruction quality and student achievement, showing the model's potential for use in automated teacher feedback tools. Our unsupervised measures show significant but weaker correlations with human labels and outcomes, and they highlight interesting linguistic patterns of funneling and focusing questions. The high performance of the supervised measure indicates its promise for supporting teachers in their instruction. 1

Revisiting the origin of, and reflections on the future of, pedagogical content knowledge

“…For example, Demszky and colleagues began with a database of transcripts of hundreds of 4th- and 5th-grade math lessons (from the Measures of Effective Teaching [MET] project; Kane & Staiger, 2012) and, using natural language processing, were able to computationally identify instances in the lessons where teachers reacted to students’ utterances in productive and generative ways (Demszky et al, 2021) and where teachers utilized instructional practices that were responsive to student thinking (Alic et al, 2022). Similarly, Suresh and colleagues (2022) created a database of transcripts from over 500 math lessons and computationally analyzed teachers’ utterances for the prevalence of “talk moves” within the widely used accountable talk framework (Michaels et al, 2008).…”

Empowering Teacher Learning with AI: Automated Evaluation of Teacher Attention to Student Ideas during Argumentation-focused Discussion

“…Advances in Artificial Intelligence (AI) and Natural Language Processing (NLP) have great potential for analyzing instructional discourse and providing substantive feedback to support teacher learning. Possible applications include identifying different types of classroom activities [13,43,44,52] and providing automated feedback on various teacher discourse moves, such as moves designed to (a) guide discussion and ensure students' participation [20,50], (b) ask authentic questions; namely, questions for which the answers are not presupposed by the teacher [1,22] or (c) restate and use student ideas [10]. Traditional approaches to automated analysis of classroom discourse typically employ supervised machine learning (ML) methods combined with manual feature engineering and human expert annotation of collected data according to evaluation rubrics.…”