Pre-College Computing Outreach Research

2019

Education Sciences

Self Cite

There has been considerable investment in pre-college educational interventions for all areas of STEM (including computer science). The goal of many of these initiatives is to engage and interest students early in their educational career. In this study, a systematic literature review was undertaken to determine the demographic and program data collected and reported for the field of computing education and for other STEM disciplines for activities that were not designed as part of the formal in-class curriculum (e.g., outreach activities). A comparison-contrast analysis of the resulting 342 articles found similarities and key differences in the reporting of this data as well as overarching characteristics of missing or incomplete reporting across disciplines. Authors from both fields reported equally well in the four categories studied: information about evaluation, participant gender, participant race and/or ethnicity, and activity demographics. However, the computing education articles were more likely to have clearly stated research questions and comparative analysis based on demographic characteristics. They were less likely to include the number of participants in the study, participant age/grade level, socioeconomic status, disability information, location of intervention, and instructor demographics. Through this analysis, it was determined that reporting can be improved across all disciplines to improve the quantity of data needed to replicate studies and to provide complete data sets that provide for the comparison of collected data.

Section: Synthesis Of Results Study Bias Limitationsmentioning

confidence: 99%

Section: Location Where the Invention Took Placementioning

confidence: 93%

See 1 more Smart Citation

A Systematic Review Exploring the Differences in Reported Data for Pre-College Educational Activities for Computer Science, Engineering, and Other STEM Disciplines

2019

Education Sciences

Self Cite

“…Although still in its infancy compared to other STEM fields, K-12 computing education research is carefully being tracked for the purposes of empirical investigation [13,29]. This is not unlike computing at the undergraduate level [5,15,24] and the traditional sciences [8,23] which report on challenges, trends, and recommendations in these educational fields.…”

Section: Introductionmentioning

confidence: 99%

A Longitudinal Analysis of K-12 Computing Education Research in the United States

Upadhyaya

Proceedings of the 51st ACM Technical Symposium on Computer Science Education

2020

Self Cite

The availability of computer science education in primary and secondary schools in the United States has grown rapidly over the last decade. Computing education research in K-12 has been increasing as well. In this study, we conducted a longitudinal analysis of seven years of data (2012-2018), curated from over 500 articles across 10 publication venues to identify trends in K-12 computing education research such as geographic location and curriculum and concepts taught. The data shows a decrease in the number of studies covering K-12 students receiving computing education even while there is an increase in the number of states adopting and implementing standards. The number of different concepts being researched is increasing, potentially reflecting the growth in what is being taught in the classroom. Demographic data is underreported (e.g., socioeconomic status (SES) and disabilities of participants) which could directly limit generalizability of the studies to different learners as well as the ability to replicate and compare studies. We conclude with recommendations for how to better position this work for others trying to use the results to guide their efforts in creating standards or adopting techniques into their classrooms.

“…Within research in K-12 computing education, previous research has shown that studies and interventions are not well described, missing key variables for understanding the populations studied and the intervention its [10,12,23]). Researchers have also discovered weaknesses in the way statistics are reported and used, such as under-reporting of effect size [24,27,29].…”

Section: Introductionmentioning

confidence: 99%

A Gap Analysis of Statistical Data Reporting in K-12 Computing Education Research

Proceedings of the 51st ACM Technical Symposium on Computer Science Education

2020

Self Cite

The quality of reporting of experimental results in computing education literature has been previously shown to be less than rigorous. In this study, we first examined research standards set forth by four organizations: American Psychology Association (APA), American Educational Research Association (AERA), What Works Clearinghouse (WWC), and the CONsolidated Standards of Reporting Trials (CONSORT). We selected the most important data standards based on their prominence across all four and the most typical study designs in computing education research. We then examined 76 articles designated as quantitative research studies (K-12) published in ten venues (2012-2018) to determine whether the reporting in these articles met these five standards. Findings indicate that only 48% of these articles report effect size and even fewer (11%) report confidence intervals and levels. We found that reported data did not meet the standard that data should be "reported in a way that the reader could construct effect-size estimates and confidence intervals beyond those supplied in the paper". Additionally, authors used existing instruments less than a quarter of the time (24%) and used instruments with evidence of reliability and validity less than half of the time (39%). We conclude with recommendations for those in the K-12 computing education research community to consider when reporting statistical data in future work so that we can increase the level of rigorous reporting in this growing field.