Making Sense of Sensemaking: Understanding How K–12 Teachers and Coaches React to Visual Analytics

Abstract

With the spread of learning analytics (LA) dashboards in K--12 schools, educators are increasingly expected to make sense of data to inform instruction. However, numerous features of school settings, such as specialized vantage points of educators, may lead to different ways of looking at data. This observation motivates the need to carefully observe and account for the ways data sensemaking occurs, and how it may differ across K--12 professional roles. Our mixed-methods study reports on interviews and think-aloud sessions with middle-school mathematics teachers and instructional coaches from four districts in the United States. By exposing educators to an LA dashboard, we map their varied reactions to visual data and reveal prevalent sensemaking patterns. We find that emotional, analytical, and intentional responses inform educators’ sensemaking and that different roles at the school afford unique vantage points toward data. Based on these findings, we offer a typology for representing sensemaking in a K--12 school context and reflect on how to expand visual LA process models.

Implementation of Learning Analytics in Primary and Secondary School: A Systematic Literature Review

“…Similarly, Campos et al (2021) found that emotional, analytical, and intentional responses inform teachers' sense-making of the dashboard. In this light, teachers will respond differently to data visualization in the dashboard based on the different roles they play in schools.…”

Co‐designing teacher support technology for problem‐based learning in middle school science

“…Using a think‐aloud protocol, teachers reviewed student results and feedback provided on the RISE dashboard, interpreted what they saw, and customized class lesson plans for the next day (as they saw fit). Prior research has noted the benefits of think‐aloud protocols on tasks involving building interpretations (Charters, 2003), including providing a low‐entry barrier (Campos et al, 2021) and tracing users' thinking (Liu & Stasko, 2010). To obtain verbalizations that accurately reflected the cognitive processes teachers implemented during responsive teaching, we refrained from providing detailed instructions or interpretation of results.…”

“…It is also important to understand how teachers use interpretations of the learning analytics to adapt their pedagogical approaches (Campos et al, 2021). We hypothesize that this requires earlier engagement in responsive practices with teachers and coming to a shared understanding about the data they need to understand students open‐ended PBL strategies.…”

Designing LADs That Promote Sensemaking: A Participatory Tool

“…The sensemaking framework of instructor analytics using dashboards elaborated in [33] starts by identifying the educational questions that lead to investigation and interpretation of the dashboard, followed by pedagogical responses. The process model for LA sensemaking through dashboards described in [6] evolves from one or more pedagogical events to its multiple data representations used by a set of sensemaking sub-processes (emotional, analytical, and intentional dimensions) to produce an interpretation of the events. The phases defined by these models are similar to the levels of situational awareness (SA) investigated by the naturalistic movement to explore human decisions [12]: perception of environmental elements in a volume of time and space, comprehension of their meaning, and projection of their state in the near future.…”