An instructor dashboard for real-time analytics in interactive programming assignments

Diana, Nicholas; Eagle, Michael; Stamper, John C.; Grover, Shuchi; Bienkowski, Marie; Basu, Satabdi

doi:10.1145/3027385.3027441

Cited by 53 publications

(33 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…First, clickstream data can help instructors and practitioners understand how students are using the available resources in an effort to improve instructional designs. For instance, instructors can monitor which resources students use most and test different designs that might allow them to better calibrate the course, either to emphasize important resources that are valuable but under-utilized by students or to provide more resources that students favor, affording more targeted guidance and feedback (Bodily and Verbert, 2017;Diana et al 2017;Shi, Fu, Chen, and Qu, 2015). Second, the realtime accessibility of behavioral clickstream data can be used to develop automatic feedback and intervention modules within the LMS.…”

Section: Clickstream Data and Its Use In Higher Education Researchmentioning

confidence: 99%

The benefits and caveats of using clickstream data to understand student self-regulatory behaviors: opening the black box of learning processes

Baker

Park

et al. 2020

Int J Educ Technol High Educ

View full text Add to dashboard Cite

Student clickstream data-time-stamped records of click events in online courses-can provide fine-grained information about student learning. Such data enable researchers and instructors to collect information at scale about how each student navigates through and interacts with online education resources, potentially enabling objective and rich insight into the learning experience beyond self-reports and intermittent assessments. Yet, analyses of these data often require advanced analytic techniques, as they only provide a partial and noisy record of students' actions. Consequently, these data are not always accessible or useful for course instructors and administrators. In this paper, we provide an overview of the use of clickstream data to define and identify behavioral patterns that are related to student learning outcomes. Through discussions of four studies, we provide examples of the complexities and particular considerations of using these data to examine student self-regulated learning.

show abstract

Section: Clickstream Data and Its Use In Higher Education Researchmentioning

confidence: 99%

The benefits and caveats of using clickstream data to understand student self-regulatory behaviors: opening the black box of learning processes

Baker

Park

et al. 2020

Int J Educ Technol High Educ

View full text Add to dashboard Cite

show abstract

“…Consequently, this process will help filter out hints not related to a student's struggle before presenting them to the student. In addition, automated tinkering detection can be used in building a dashboard [4] for instructors so that they can see students' tinkering behaviors in the programming environment and provide interventions accordingly.…”

Section: Discussionmentioning

confidence: 99%

Defining Tinkering Behavior in Open-ended Block-based Programming Assignments

Dong

Marwan

Cateté

et al. 2019

Proceedings of the 50th ACM Technical Symposium on Computer Science Education

View full text Add to dashboard Cite

Tinkering has been shown to have a positive influence on students in open-ended making activities. Open-ended programming assignments in block-based programming resemble making activities in that both of them encourage students to tinker with tools to create their own solutions to achieve a goal. However, previous studies of tinkering in programming discussed tinkering as a broad, ambiguous term, and investigated only self-reported data. To our knowledge, no research has studied student tinkering behaviors while solving problems in block-based programming environments. In this position paper, we propose a definition for tinkering in block-based programming environments as a kind of behavior that students exhibit when testing, exploring, and struggling during problem-solving. We introduce three general categories of tinkering behaviors (test-based, prototype-based, and construction-based tinkering) derived from student data, and use case studies to demonstrate how students exhibited these behaviors in problem-solving. We created the definitions using a mixed-methods research design combining a literature review with data-driven insights from submissions of two open-ended programming assignments in iSnap, a block-based programming environment. We discuss the implication of each type of tinkering behavior for learning. Our study and results are the first in this domain to define tinkering based on student behaviors in a block-based programming environment. CCS CONCEPTS • Social and professional topics → Computing education; • Applied computing → Interactive learning environments;

show abstract

“…A learner can demonstrate the acquisition of knowledge and skills as a product of their learning process. Most key studies in this category operationalize affected learning through grades or test scores or scores [29], [33]- [35], [37], [49], [50], [53], [54], [57], [61], [63], [67]- [70], [74]- [76], which is a direct assessment of learning as performance on a task (e.g., an exam or final test) [94]. Although grades may seem to be a direct operational definition, this is debatable.…”

Section: Analysis and Refinement Of The Classification Schemementioning

confidence: 99%

Where is the Learning in Learning Analytics? A Systematic Literature Review on the Operationalization of Learning-Related Constructs in the Evaluation of Learning Analytics Interventions

Knobbout

Stappen

2020

IEEE Trans. Learning Technol.

View full text Add to dashboard Cite

Learning technologies enable interventions in the learning process aiming to improve learning. Learning analytics provides such interventions based on analysis of learner data, which are believed to have beneficial effects on both learning and the learning environment. Literature reporting on the effects of learning analytics interventions on learning allows us to assess in what way learning analytics improves learning. No standard set of operational definitions for learning affected by learning analytics interventions is available. We performed a systematic literature review of 1932 search hits, which yielded 62 key studies. We analyzed how affected learning was operationalized in these key studies and classified operational definitions into three categories: 1) learning environment; 2) learning process; and 3) learning outcome. A deepening analysis yielded a refined classification scheme with 11 subcategories. Most of the analyzed studies relate to either learning outcome or learning process. Only nine of the key studies relate to more than one category. Given the complex nature of applying learning analytics interventions in practice, measuring the effects on a wider spectrum of aspects can give more insight into the workings of learning analytics interventions on the different actors, processes, and outcomes involved. Based on the results of our review, we recommend making deliberate decisions on the (multiple) aspects of learning one tries to improve by applying learning analytics. Our refined classification with examples of operational definitions may help both academics and practitioners doing so, as it allows for a more structured, grounded, and comparable positioning of learning analytics benefits.

show abstract

An instructor dashboard for real-time analytics in interactive programming assignments

Cited by 53 publications

References 15 publications

The benefits and caveats of using clickstream data to understand student self-regulatory behaviors: opening the black box of learning processes

The benefits and caveats of using clickstream data to understand student self-regulatory behaviors: opening the black box of learning processes

Defining Tinkering Behavior in Open-ended Block-based Programming Assignments

Where is the Learning in Learning Analytics? A Systematic Literature Review on the Operationalization of Learning-Related Constructs in the Evaluation of Learning Analytics Interventions

Contact Info

Product

Resources

About