When Data Teams Struggle: Learning from Less Successful Data Use Efforts

Abstract

Because learning from failures is just as important as learning from successes, we used qualitative case study data gathered in the Netherlands and the United States to examine instances in which data teams struggle to contribute to school improvement. Similar factors in both the Dutch and U.S. case hindered the work of the data teams, such as data use focused on accountability instead of improvement. Some hindering factors were more context-specific. Learning why, how, and when data use efforts do not go smoothly provides an important contribution to research and practice.

Exploratory Data Analysis in Schools: A Logic Model to Guide Implementation

“…A 2015 report released by the Bill and Melinda Gates Foundation found that current data systems are often overwhelming to teachers due to the large amounts of data from multiple sources and the frequent incompatibility of tools and data sets. Other research previously discussed in this article examines similar challenges (Dhar, 2013;Schildkamp & Datnow, 2020;Tichnor-Wagner et al, 2017). The beauty of EDA is that it can yield quick insights with little skill, as it is process-focused rather than theory-focused (Tukey, 1977).…”

“…As with any continuous improvement initiative, school leaders must foster a culture in which EDA can thrive. Data use must be a regular part of the school' s continuous improvement conversation and be embedded in the underlying beliefs of the school before it can become meaningful to staff (Day & Sammons, 2013;Gerzon & Guckenburg, 2015;Schildkamp & Datnow, 2020). The first step in this process is to approach data work with an open and collaborative mindset.…”

Facilitating collaborative reflective inquiry amongst teachers: What do we currently know?

“…of the curriculum); practitioners seeking new knowledge; practitioners' tacit knowledge constantly converted into shared knowledge through interaction; and practitioners applying new ideas and information to problem solving and solutions addressing the needs of, for example, students (Sebba et al, 2012;Stoll et al, 2006). As a result, RPI is considered more effective when it involves a high level of 'depth of inquiry'; with depth of inquiry defined as the degree to which practitioners employ higher level thinking skills when engaged in actions such as analysis, synthesis, goal setting, and reflection (Opfer & Pedder, 2011;Schildkamp & Datnow, 2020;Schildkamp, Poortman, & Handelzalts, 2016).…”

“…In education, when teachers display high depth of inquiry, they tend to be engaged in meaning making and the development new knowledge based on evidence and focused on the planning of actions to improve teaching and learning (Rogers, 2002). Practitioners that engage in lower levels of depth of inquiry are typically focused more on sharing information, storytelling and describing existing actions (Schildkamp & Datnow, 2020;Vanlommel, Van Gasse, Vanhoof, & Van Petegem, 2017). In other words, with higher 'depth of inquiry', practitioners seek to further their understanding and turn this into action rather than simply transmit 'what is known'.…”

“…Furthermore, when practitioners display high levels of 'depth of inquiry', evidence is used conceptually and/or instrumentally as opposed to strategically or symbolically. Evidence is employed to change teachers' and school leaders' thinking (something referred to as conceptual evidence use) and used to make actual changes in classrooms and school (instrumental evidence use) (Farley-Ripple, May, Karpyn, Tilley, & McDonough, 2018;Schildkamp & Datnow, 2020). This division between conceptual and instrumental is also reflected by Sparks-Langer (e.g.…”