A Systematic Investigation of Replications in Computing Education Research

Hao, Qiang; Smith, David H.; Iriumi, Naitra; Tsikerdekis, Michail; Ko, Amy J.

doi:10.1145/3345328

Cited by 39 publications

(27 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The present work responds to these needs. More broadly, we answer recent calls from the CER literature for more replication studies [19] and more work towards standardized instruments of measurement [39].…”

Section: Research Questionsmentioning

confidence: 71%

Measuring the Cognitive Load of Learning to Program: A Replication Study

Zavgorodniaia

Duran

Hellas

et al. 2020

United Kingdom &Amp; Ireland Computing Education Research Conference.

View full text Add to dashboard Cite

Cognitive load (CL) on a learner's working memory has emerged as an influential concept in computing education and beyond. CL is commonly divided in at least two components, intrinsic load (IL) and extraneous load (EL). We seek progress on two questions: (1) How can CL components be measured in the programming domain?(2) How should CL measurement deal with the "third component" of germane load (GL)? We replicate two studies: Morrison and colleagues' [49] evaluation of a questionnaire for self-assessing CL in programming, which is an adaptation of a generic instrument; and Jiang and Kalyuga's [24] study, which found support for a twocomponent measure of CL in language learning, with GL redundant. We crowd-sourced CL data using Morrison's questions at the end of a video tutorial on programming for beginners. A confirmatory factor analysis found strong support for a three-factor model, with factors matching the items intended to capture IL, EL, and GL, respectively. A two-factor model with IL-targeting and GL-targeting items combined gave a poorer fit. Our findings strengthen the claims of discriminant validity and internal reliability for Morrison's CL questionnaire for programming; construct validity for GL remains open, however. We affirm the need for further research on the twocomponent theory of CL and the sensitivity of CL self-assessments to contextual factors. CCS CONCEPTS• Social and professional topics → Computing education.

show abstract

Section: Research Questionsmentioning

confidence: 71%

Measuring the Cognitive Load of Learning to Program: A Replication Study

Zavgorodniaia

Duran

Hellas

et al. 2020

United Kingdom &Amp; Ireland Computing Education Research Conference.

View full text Add to dashboard Cite

show abstract

“…We also replicate, by extension, the modality studies in other domains that Morrison replicated. In doing so, we respond to requests for more replication studies in CER [35] and for more varied replications of multimedia-learning principles in different domains and with learners other than the typical cohort of young college-student volunteers [64,Chapter 20].…”

Section: H2bmentioning

confidence: 99%

Should Explanations of Program Code Use Audio, Text, or Both? A Replication Study

Zavgorodniaia

Hellas

Seppälä

et al. 2020

Koli Calling '20: Proceedings of the 20th Koli Calling International Conference on Computing Education Research

View full text Add to dashboard Cite

Studies in educational psychology suggest that people learn better when visual learning materials are accompanied by audio explanations rather than textual ones. Research on how this modality effect applies to computing education is scarce and inconclusive. We explore whether modality of instruction affects learning from videos that use a series of example programs to explain how variables work in Python. Learners (n=186) were crowdsourced from the internet and randomized in three groups, who received explanations as audio, text, or both, respectively. We did not find significant differences between the groups in near transfer to code-tracing tasks or perceived cognitive load. The result affirms the need to further investigate instructional modalities in programming education. There are a number of theoretical, methodological, and instructional-design factors that may explain these and earlier findings; we trace out future research that could explore those factors. CCS CONCEPTS• Social and professional topics → Computing education.

show abstract

“…This is problematic, as central premises of subsequent studies are occasionally based on the results of the fundamental studies. The lack of replication studies in computing education in general has only recently received scientific attention [53]. Partly because of the lack of replication, we argue that educational SQL research is not mature enough to distinguish between patterns and anti-patterns.…”

Section: Replicationmentioning

confidence: 89%

SQL Education

Taipalus

Seppänen

2020

ACM Trans. Comput. Educ.

View full text Add to dashboard Cite

Structured Query Language (SQL) skills are crucial in software engineering and computer science. However, teaching SQL effectively requires both pedagogical skill and considerable knowledge of the language. Educators and scholars have proposed numerous considerations for the betterment of SQL education, yet these considerations may be too numerous and scattered among different fora for educators to find and internalize, as no systematic mappings or literature reviews regarding SQL education have been conducted. The two main goals of this mapping study are to provide an overview of educational SQL research topics, research types and publication fora, and to collect and propagate SQL teaching practices for educators to utilize. Additionally, we present a short future research agenda based on insights from the mapping process. We conducted a systematic mapping study complemented by snowballing techniques to identify applicable primary studies. We classified the primary studies according to research type, and utilized directed content analysis to classify the primary studies by their topic. Out of our selected 89 primary studies, we identified six recurring topics: (i) student errors in query formulation; (ii) characteristics and presentation of the exercise database; (iii) specific and (iv) non-specific teaching approach suggestions; (v) patterns and visualization; and (vi) easing teacher workload. We list 66 teaching approaches the primary studies argued for (and in some cases against). For researchers, we provide a systematic map of educational SQL research, and future research agenda. For educators, we present an aggregated body of knowledge on teaching practices in SQL education over a time frame of 30 years. In conclusion, we suggest that replication studies, studies on advanced SQL concepts, and studies on aspects other than data retrieval are needed to further educational SQL research.

show abstract

A Systematic Investigation of Replications in Computing Education Research

Cited by 39 publications

References 31 publications

Measuring the Cognitive Load of Learning to Program: A Replication Study

Measuring the Cognitive Load of Learning to Program: A Replication Study

Should Explanations of Program Code Use Audio, Text, or Both? A Replication Study

SQL Education

Contact Info

Product

Resources

About