Evidence That Computer Science Grades Are Not Bimodal

Abstract

Although it has never been rigourously demonstrated, there is a common belief that CS grades are bimodal. We statistically analyzed 778 distributions of final course grades from a large research university, and found only 5.8% of the distributions passed tests of multimodality. We then devised a psychology experiment to understand why CS educators believe their grades to be bimodal. We showed 53 CS professors a series of histograms displaying ambiguous distributions and asked them to categorize the distributions. A random half of participants were primed to think about the fact that CS grades are commonly thought to be bimodal; these participants were more likely to label ambiguous distributions as "bimodal". Participants were also more likely to label distributions as bimodal if they believed that some students are innately predisposed to do better at CS. These results suggest that bimodal grades are instructional folklore in CS, caused by confirmation bias and instructor beliefs about their students.

Exploring Bimodality in Introductory Computer Science Performance Distributions

“…Most of the existing evidence for and against the double hump comes from the test for programming aptitude proposed by Dehnadi and Bornat (2006). A smaller set of studies uses actual performance data such as final grades from computer science courses (Patitsas, Berlin, Craig, & Easterbrook, 2016). Computer science course performance data provide a real-life context for examining the double hump phenomenon.…”

“…To address the overarching research question, we analyzed the data in the spirit of Patitsas et al (2016) and provide the following statistics: kurtosis, skewness, Shapiro-Wilk Test of Normality, and Hartigan's dip test statistic (Hartigan & Hartigan, 1985) for unimodality/multimodality. The values for kurtosis, skewness, Shapiro-Wilk Test of Normality were calculated using SPSS.…”

“…Ahadi and Lister (2013), who examined four tests from an introductory programming course, highlighted the complexity of accounting for the bimodal grade distribution and noted that: "advocates of the various hypotheses-Geek Genes, Prior Knowledge, Stumbling Points and Learning Edge Momentum -can all find support for their respective hypotheses, in aspects of the data in this paper" (p. 126). Using 778 distributions of final course grades, Patitsas, Berlin, Craig, and Easterbrook (2016) found that only 5.8% of the distributions passed tests of multimodality. Moreover it is common to find general statements and anecdotes in the literature about the bimodality phenomenon without relevant evidence.…”

“…The anecdotal statement is accepted on face value and no further supporting evidence is provided. Furthermore, Patitsas, Berlin, Craig, and Easterbrook (2016) noted that prior work on the bimodality distribution phenomenon in CS performance has generally lacked robust statistical testing, with many resorting to visual inspections of distributions to assess bimodality (Lister, 2010). Hence, there is a need to further understand the topic by conducting robust statistical testing on actual performance data from computer science courses.…”

Growth Mindset in Computational Thinking Teaching and Teacher Training

“…Learning to program may appear a too challenging task, achievable only from those with a so called "geek gene" [10]. Moreover, stereotypes lead to consider computer scientists as singularly focused, asocial, competitive, male gures [9].…”