Assisting software engineering students in analyzing their performance in software development

Raza, Mushtaq; Faria, João Pascoal; Salazar, Rafael

doi:10.1007/s11219-018-9433-7

Cited by 10 publications

(11 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While colour‐coded messages (Baneres et al, 2019; Broos, Verbert, et al, 2017) and traffic light metaphors (Cha & Park, 2019; Raza et al, 2019; Ullmann et al, 2019) represented the most frequent types of designs; other types of visualisation elements such as social network representations (Vovides & Inman, 2016), plant images (Muldner et al, 2015), timeline (Sedrakyan et al, 2017), avatars (Charleer et al, 2016) and speedometer metaphor (Michel et al, 2017) were also used.…”

Section: Resultsmentioning

confidence: 99%

“…Based on the coded data, there is a considerable lack of alignment between LADs’ target outcomes and the domain measures used to evaluate their achievement. For example, although 20 of the LAD target outcomes (71%) related to regulation of cognition, only 6 out of these 20 LADs (30%) were actually evaluated using cognitive domain measures (eg, learning outcomes) that reflect the efficacy of the LADs relative to the stated target outcome (Aljohani et al, 2019; Kim et al, 2016; Lkhagvasuren et al, 2016; Raza et al, 2019; Seanosky et al, 2017; Van Horne et al, 2018).…”

Section: Resultsmentioning

confidence: 99%

“…There is, however, a push for greater integration between OLMs and LADs based not only on their shared tradition as data visualisation tools, but also based on their progressive convergent developments: while some OLMs have been integrating behavioural metrics as part of their design approach (Bodily et al, 2018; Bull, 2020), some LADs have been using learner modelling (Baneres et al, 2019; Raza et al, 2019). For example, Baneres et al, 2019 have used an adaptive predictive model to identify students at‐risk in the development of an early warning system that also included visualisation dashboard and an e‐mail messaging system to send personalised recommendations based on student profile.…”

Section: Discussionmentioning

confidence: 99%

See 2 more Smart Citations

Staying on target: A systematic literature review on learner‐facing learning analytics dashboards

Valle

Antonenko

Dawson

et al. 2021

Brit J Educational Tech

View full text Add to dashboard Cite

The advances in technology to capture and process unprecedented amounts of educational data has boosted the interest in Learning Analytics Dashboard (LAD) applications as a way to provide meaningful visual information to administrators, parents, teachers and learners. Despite the frequent argument that LADs are useful to support target users and their goals to monitor and act upon the information provided, little is known about LADs’ theoretical underpinnings and the alignment (or lack thereof) between LADs intended outcomes and the measures used to evaluate their implementation. However, this knowledge is necessary to illuminate more efficient approaches in the development and implementation of LAD tools. Guided by the self‐regulated learning perspective and using the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) framework, this systematic literature review addressed this gap by examining whether and how learner‐facing LAD’s target outcomes align with the domain measures used to evaluate their implementations. Out of the 1297 papers retrieved from 15 databases, 28 were included in the final quantitative and qualitative analysis. Results suggested an intriguing lack of alignment between LADs’ intended outcomes (mostly cognitive domain) and their evaluation (mostly affective measures). Based on these results and on the premise that LADs are designed to support learners, a critical recommendation from this study is that LADs’ target outcomes should guide the selection of measures used to evaluate the efficacy of these tools. This alignment is critical to enable the construction of more robust guidelines to inform future endeavours in the field. What is already known about this topic There has been an increased interest and investment in learning analytics dashboards to support learners as end‐users. Learner‐facing learning analytics dashboards are designed with different purposes, functionalities and types of data in an attempt to influence learners’ behaviour, achievement and skills. What this paper adds This paper reports trends and opportunities regarding the design of learner‐facing learning analytics dashboards, contexts of implementation, as well as types and features of learner‐facing learning analytics dashboard studies. The paper discusses how affect and motivation have been largely overlooked as target outcomes in learner‐facing learning analytics dashboards. Implications for practice and/or policy Based on the evidence gathered through the review, this paper makes recommendations for theory (eg, inclusion of motivation as an important target outcome). The paper makes recommendations related to the design, implementation and evaluation of learning analytics dashboards. The paper also highlights the need for further integration between learner‐facing learning analytics dashboards and open learner models.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Staying on target: A systematic literature review on learner‐facing learning analytics dashboards

Valle

Antonenko

Dawson

et al. 2021

Brit J Educational Tech

View full text Add to dashboard Cite

show abstract

“…In the second experiment [10], we conducted a controlled experiment with 61 students that attended the 2016 edition of the course, to assess if students that use ProcessPAIR for performing their final report assignment are more satisfied with the tool support (RQ2.1), produce higher quality reports (RQ2.2), and spend less time (RQ2.3) than students that perform that assignment in a traditional way (using Process Dashboard only).…”

Section: B Controlled Experimentsmentioning

confidence: 99%

“…In previous publications, we presented the derivation of an example performance model [8] (without automatic calibration), a previous tool prototype [9], and a validation experiment [10]. The specific contribution of this article is the presentation of an inside view of the ProcessPAIR method, detailing its main steps (model definition, model calibration, and performance analysis), with the help of (meta)models and examples.…”

Section: Introductionmentioning

confidence: 99%

The ProcessPAIR Method for Automated Software Process Performance Analysis

Raza

Faria

2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

High-maturity software development processes and development environments with automated data collection can generate significant amounts of data that can be periodically analyzed to identify performance problems, determine their root causes, and devise improvement actions. However, conducting the analysis manually is challenging because of the potentially large amount of data to analyze, the effort and expertise required, and the lack of benchmarks for comparison. In this article, we present ProcessPAIR, a novel method with tool support designed to help developers analyze their performance data with higher quality and less effort. Based on performance models structured manually by process experts and calibrated automatically from the performance data of many process users, it automatically identifies and ranks performance problems and potential root causes of individual subjects, so that subsequent manual analysis for the identification of deeper causes and improvement actions can be appropriately focused. We also show how ProcessPAIR was successfully instantiated and used in software engineering education and training, helping students analyze their performance data with higher satisfaction (by 25%), better quality of analysis outcomes (by 7%), and lower effort (by 4%), as compared to a traditional approach (with reduced tool support).

show abstract

Proposal of a Support System for Visualization of Acquired Skills Using iCD in Project-Based Learning for Software Development and How to Acquire its Data

Kondo,

Hashiura,

Hazeyama

2023

2023 IEEE 35th International Conference on Software Engineering Education and Training (CSEE&T)

View full text Add to dashboard Cite

Assisting software engineering students in analyzing their performance in software development

Cited by 10 publications

References 13 publications

Staying on target: A systematic literature review on learner‐facing learning analytics dashboards

Staying on target: A systematic literature review on learner‐facing learning analytics dashboards

The ProcessPAIR Method for Automated Software Process Performance Analysis

Proposal of a Support System for Visualization of Acquired Skills Using iCD in Project-Based Learning for Software Development and How to Acquire its Data

Contact Info

Product

Resources

About