A Framework for Using Hypothesis-Driven Approaches to Support Data-Driven Learning Analytics in Measuring Computational Thinking in Block-Based Programming Environments

Abstract

Systematic endeavors to take computer science (CS) and computational thinking (CT) to scale in middle and high school classrooms are underway with curricula that emphasize the enactment of authentic CT skills, especially in the context of programming in block-based programming environments. There is, therefore, a growing need to measure students’ learning of CT in the context of programming and also support all learners through this process of learning computational problem solving. The goal of this research is to explore hypothesis-driven approaches that can be combined with data-driven ones to better interpret student actions and processes in log data captured from block-based programming environments with the goal of measuring and assessing students’ CT skills. Informed by past literature and based on our empirical work examining a dataset from the use of the Fairy Assessment in the Alice programming environment in middle schools, we present a framework that formalizes a process where a hypothesis-driven approach informed by Evidence-Centered Design effectively complements data-driven learning analytics in interpreting students’ programming process and assessing CT in block-based programming environments. We apply the framework to the design of Alice tasks for high school CS to be used for measuring CT during programming.

Persistence in a Game-Based Learning Environment: The Case of Elementary School Students Learning Computational Thinking

“…Using this method, researchers have studied how learners’ actions are connected to their achievements, as well as to their level of involvement and persistence (Berland, Baker, & Blikstein, 2014). In the context of computer science education, such methods have been used to analyze students’ programming activity over time (Berland, Martin, Benton, Petrick Smith, & Davis, 2013; Blikstein, 2011; Boutnaru & Hershkovitz, 2015; Eguíluz, Guenaga, Garaizar, & Olivares-Rodríguez, 2017; Grover et al., 2017; Hershkovitz et al., 2019; Lu, Huang, Huang, & Yang, 2017; Nutbrown & Higgins, 2016). We continue this line of research by applying learning analytics methods to the study of CT.…”

Combining Assessment Tools for a Comprehensive Evaluation of Computational Thinking Interventions

“…These latter tools provide valuable data and learning analytics from which cognitive processes of the subject can be inferred, and they are especially useful to detect gaps and misconceptions while acquiring computational concepts. It can be highlighted the research done by the team of Shuchi Grover (Grover et al, 2017;Grover, Bienkowski, Niekrasz, & Hauswirth, 2016) in the Blockly environment, and the one from Eguiluz, Guenaga, Garaizar, and Olivares-Rodriguez (2017) using Kodetu. • CT skill transfer tools: Their objective is to assess to what extent the students are able to transfer their CT skills onto different kinds of problems, contexts, and situations.…”

Reflecting on Computational Thinking Studies for High School Education

“…The authors adopted Behavioral Identification Form to measure abstract thinking. A study by [74] proposes a framework for measuring CT in block-based programming environment Alice. The framework was applied to analyze a dataset of performance assessment task from 'Fairy Assessment in Alice,' a programming-based assessment developed by [75], to gain an understanding of students' programming process.…”