Approaches to Assess Computational Thinking Competences Based on Code Analysis in K-12 Education: A Systematic Mapping Study

Alves, Nathalia da Cruz; Wangenheim, Christiane Gresse von; Hauck, Jean Carlo Rossa

doi:10.15388/infedu.2019.02

Cited by 48 publications

(26 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…It enables an analysis of the code of App Inventor programs supported by a free web-based tool providing feedback to students and teachers in the form of a score with respect to algorithms & programming and the graphical user interface design of the apps created. The model has been developed based on a systematic mapping study [53] following an instructional design process [54] and the procedure for rubric definition proposed by Goodrich [55]. The rubric is based on the K-12 Computer Science Framework [5] as well as other rubrics and frameworks, including [21][8] [56].…”

Section: Codemaster Rubricmentioning

confidence: 99%

An Item Response Theory Analysis of Algorithms and Programming Concepts in App Inventor Projects

Alves

Wangenheim

Hauck

et al. 2021

Anais Do I Simpósio Brasileiro De Educação Em Computação (EDUCOMP 2021)

Self Cite

View full text Add to dashboard Cite

Computing education is often introduced in K-12 focusing on algorithms and programming concepts using block-based programming environments, such as App Inventor. Yet, learning programming is a complex process and novices struggle with several difficulties. Thus, to be effective, instructional units need to be designed regarding not only the content but also its sequencing taking into consideration difficulties related to the concepts and the idiosyncrasies of programming environments. Such systematic sequencing can be based on large-scale project analyses by regarding the volition, incentive, and opportunity of students to apply the relevant program constructs as latent psychometric constructs using Item Response Theory to obtain quantitative ?difficulty? estimates for each concept. Therefore, this article presents the results of a large-scale data-driven analysis of the demonstrated use in practice of algorithms and programming concepts in App Inventor. Based on a dataset of more than 88,000 App Inventor projects assessed automatically with the ANON rubric, we perform an analysis using Item Response Theory. The results demonstrate that the easiness of some concepts can be explained by their inherent characteristics, but also due to the characteristics of App Inventor as a programming environment. These results can help teachers, instructional and curriculum designers in the sequencing, scaffolding and assessment design of programming education in K-12.

show abstract

Section: Codemaster Rubricmentioning

confidence: 99%

An Item Response Theory Analysis of Algorithms and Programming Concepts in App Inventor Projects

Alves

Wangenheim

Hauck

et al. 2021

Anais Do I Simpósio Brasileiro De Educação Em Computação (EDUCOMP 2021)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Additionally, automated analyses of computational artifacts can make the assessment of large amounts of productions possible, using comparable parameters at scale. Recent studies using different programming languages provide evidence that learning analytics techniques can be used to promote a better understanding of how learners evolve as they develop programming and CT skills (Alves et al, 2019;Blikstein et al, 2014;Dasgupta and Hill, 2017;Von Wangenheim et al, 2018). In particular, different tools have been developed to analyze, assess, and give feedback to students working on the Scratch programming environment.…”

Section: Automatic Assessment Of Computational Thinkingmentioning

confidence: 99%

A data-driven approach to assess computational thinking concepts based on learners’ artifacts

Eloy¹,

Achutti²,

Fernandez³

et al. 2021

Informatics in Education

View full text Add to dashboard Cite

Integrating computational thinking into K-12 Education has been a widely explored topic in recent years. Particularly, effective assessment of computational thinking can support the understanding of how learners develop computational concepts and practices. Aiming to help advance research on this topic, we propose a data-driven approach to assess computational thinking concepts, based on the automatic analysis of data from learners' computational artifacts. As a proof of concept, the approach was applied to a Massive Open Online Course (MOOC) to investigate the course's effectiveness as well as to identify points for improvement. The data analyzed consists of over 3300 projects from the course participants, using the Scratch programming language. From that sample, we found patterns in how computational thinking manifests in projects, which can be used as evidence to guide opportunities for improving course design, as well as insights to support further research on the assessment of computational thinking.

show abstract

“…In accordance with the analysis questions, data has been collected based on the source code of App Inventor projects using the CodeMaster rubric. The CodeMaster rubric has been developed based on a systematic mapping study (Alves, Gresse von Wangenheim, & Hauck, 2019) following an instructional design process (Branch, 2010) and the procedure for rubric definition proposed by Goodrich (1997). The rubric is based on the K-12 Computer Science Framework (CSTA, 2016) as well as other rubrics and frameworks, including (Brennan & Resnick, 2012;Grover, Basu, & Schank, 2018;Sherman & Martin, 2015).…”

Section: Operationalization Of the Measurementmentioning

confidence: 99%

A Detailed Item Response Theory Analysis of Algorithms and Programming Concepts in App Inventor Projects

Alves

Wangenheim

Hauck

et al. 2021

RBIE

View full text Add to dashboard Cite

Teaching computing in K-12 is often introduced focusing on algorithms and programming concepts using block-based programming environments, such as App Inventor. Yet, learning programming is a complex process and novices struggle with several difficulties. Thus, to be effective, instructional units need to be designed regarding not only the content but also its sequencing taking into consideration difficulties related to the concepts and the idiosyncrasies of programming environments. Such systematic sequencing can be based on large-scale project analyses by regarding the volition, incentive, and opportunity of students to apply the relevant program constructs as latent psychometric constructs using Item Response Theory to obtain quantitative ‘difficulty’ estimates for each concept. Therefore, this article presents the results of a large-scale data-driven analysis of the demonstrated use in practice of algorithms and programming concepts in App Inventor. Based on a dataset of more than 88,000 App Inventor projects assessed automatically with the CodeMaster rubric, we perform an analysis using Item Response Theory. The results demonstrate that the easiness of some concepts can be explained by their inherent characteristics, but also due to the characteristics of App Inventor as a programming environment. These results can help teachers, instructional and curriculum designers in the sequencing, scaffolding, and assessment design of programming education in K-12.

show abstract

Approaches to Assess Computational Thinking Competences Based on Code Analysis in K-12 Education: A Systematic Mapping Study

Cited by 48 publications

References 53 publications

An Item Response Theory Analysis of Algorithms and Programming Concepts in App Inventor Projects

An Item Response Theory Analysis of Algorithms and Programming Concepts in App Inventor Projects

A data-driven approach to assess computational thinking concepts based on learners’ artifacts

A Detailed Item Response Theory Analysis of Algorithms and Programming Concepts in App Inventor Projects

Contact Info

Product

Resources

About