Biases and differences in code review using medical imaging and eye-tracking: genders, humans, and machines

Huang, Yu; Leach, Kevin; Sharafi, Zohreh; McKay, Nicholas; Santander, Tyler; Weimer, Westley

doi:10.1145/3368089.3409681

Cited by 22 publications

(10 citation statements)

References 107 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To properly cover a wide range of programming expertise, we were forced to give up on maintaining gender balance at each expertise level. However, several fMRI studies have reported possible gender differences in behavior, cognitive function, and neuroimaging data (David et al, 2018;Huang et al, 2020). The results obtained via this study might be biased by the gender imbalance of the subject population.…”

Section: Limitations Of the Studymentioning

confidence: 55%

Expert Programmers Have Fine-Tuned Cortical Representations of Source Code

Ikutani

Kubo

Nishida

et al. 2020

eNeuro

View full text Add to dashboard Cite

Expertise enables humans to achieve outstanding performance on domain-specific tasks, and programming is no exception. Many studies have shown that expert programmers exhibit remarkable differences from novices in behavioral performance, knowledge structure, and selective attention. However, the underlying differences in the brain of programmers are still unclear. We here address this issue by associating the cortical representation of source code with individual programming expertise using a data-driven decoding approach. This approach enabled us to identify seven brain regions, widely distributed in the frontal, parietal, and temporal cortices, that have a tight relationship with programming expertise. In these brain regions, functional categories of source code could be decoded from brain activity and the decoding accuracies were significantly correlated with individual behavioral performances on a source-code categorization task. Our results suggest that programming expertise is built upon fine-tuned cortical representations specialized for the domain of programming. Significance StatementThe expertise needed for programming has attracted increasing interest among researchers and educators in our computerized world. Many studies have demonstrated that expert programmers exhibit superior behavioral performance, knowledge structure, and selective attention; but how their brain accommodates such superiority is not well understood. In this paper we have recorded brain activities from subjects covering a wide range of programming expertise. The results show that functional categories of source code can be 1 decoded from their brain activity and the decoding accuracies on the seven brain regions in frontal, parietal, temporal cortices are significantly correlated with individual behavioral performances. This study provides evidence that outstanding performances of expert programmers are associated with domain-specific cortical representations in these widely distributed brain areas.

show abstract

Section: Limitations Of the Studymentioning

confidence: 55%

Expert Programmers Have Fine-Tuned Cortical Representations of Source Code

Ikutani

Kubo

Nishida

et al. 2020

eNeuro

View full text Add to dashboard Cite

show abstract

“…Moreover, the authors suggest that women are as productive as their male colleagues in an inclusive OSS project, with some women developers demonstrating more productivity than the average men developers. Even not supporting any inferences about whether men or women are more accurate at code review, results from Huang et al [PS14]'s study indicates women present more reliable activity patterns.…”

Section: Discussionmentioning

confidence: 78%

“…According to the GitHub dataset of six projects evaluated by Paul et al [PS13], women are more likely to write reviews expressing sentiments in the text to another woman than to a man during code reviews. Huang et al [PS14] used medical imaging and eye-tracking to evaluate the visual and cognitive processes and patterns of neural activation followed by reviewers while performing code reviews. Authors found that women spent significantly more time analyzing pull-request messages and author pictures (regardless of their identity) than the code itself when performing code reviews.…”

Section: Types Of Contributions That Women Make In Oss Projectsmentioning

confidence: 99%

“…Even submitting fewer pull-requests, according to Kofink [PS10], women have 64.2% of merge acceptance rate, nearly equivalent to men's (63.9%), and continue to have high acceptance rates as they gain experience. According to Huang et al [PS14], humans (non-bots) are 4.7% more likely to accept pull-requests submitted by women than men. On the other hand, Terrell et al [PS4] found that women have a 12% lower pull-request acceptance rate when they explicitly identify themselves as women, comparable to 3.8% for men who disclosure their gender.…”

Section: Rate Of Acceptance Of Women's Contributions In Oss Projectsmentioning

confidence: 99%

See 1 more Smart Citation

Women's Participation in Open Source Software: A Survey of the Literature

Trinkenreich¹,

Wiese²,

Sarma³

et al. 2021

Preprint

View full text Add to dashboard Cite

Participation of women in Open Source Software (OSS) is very unbalanced, despite various efforts to improve diversity. This is concerning not only because women do not get the chance of career and skill developments afforded by OSS, but also because OSS projects suffer from a lack of diversity of thoughts because of a lack of diversity in their projects. Studies that characterize women's participation and investigate how to attract and retain women are spread across multiple fields, including information systems, software engineering, and social science. This paper systematically maps, aggregates, and synthesizes the state-of-the-art on women's participation in Open Source Software. It focuses on women's representation and the demographics of women who contribute to OSS, how they contribute, the acceptance rates of their contributions, their motivations and challenges, and strategies employed by communities to attract and retain women. We identified 51 articles (published between 2005 and 2021) that investigate women's participation in OSS. According to the literature, women represent about 9.8% of OSS contributors; most of them are recent contributors, 20-37 years old, devote less than 5h/week to OSS, and make both non-code and code contributions. Only 5% of projects have women as core developers, and women author less than 5% of pull-requests but have similar or even higher rates of merge acceptance than men.Besides learning new skills and altruism, reciprocity and kinship are motivations especially relevant for women, but can leave if they are not compensated for their contributions. Women's challenges are mainly social, including lack of peer parity and non-inclusive communication from a toxic culture. The literature reports ten strategies, which were mapped to six of the seven challenges. Based on these results, we provide guidelines for future research and practice. CCS Concepts: • Software and its engineering → Software evolution; • Human-centered computing → Open source software; • Social and professional topics → Women.

show abstract

“…Another group of studies addresses developers' cognitive biases that might affect code review outcome [20,35,50,54]. Huang et al investigated how cognitive biases relate to code review process in a controlled experiment using medical imaging and eye-tracking.…”

Section: Cognitive Aspects In Code Reviewmentioning

confidence: 99%

Artefacts Package - First come first served: the impact of file position on code review

Fregnan

Braz

D’Ambros

et al. 2022

View full text Add to dashboard Cite

The most popular code review tools (e.g., Gerrit and GitHub) present the files to review sorted in alphabetical order. Could this choice or, more generally, the relative position in which a file is presented bias the outcome of code reviews? We investigate this hypothesis by triangulating complementary evidence in a two-step study.First, we observe developers' code review activity. We analyze the review comments pertaining to 219,476 Pull Requests (PRs) from 138 popular Java projects on GitHub. We found files shown earlier in a PR to receive more comments than files shown later, also when controlling for possible confounding factors: e.g., the presence of discussion threads or the lines added in a file. Second, we measure the impact of file position on defect finding in code review. Recruiting 106 participants, we conduct an online controlled experiment in which we measure participants' performance in detecting two unrelated defects seeded into two different files. Participants are assigned to one of two treatments in which the position of the defective files is switched. For one type of defect, participants are not affected by its file's position; for the other, they have 64% lower odds to identify it when its file is last as opposed to first. Overall, our findings provide evidence that the relative position in which files are presented has an impact on code reviews' outcome; we discuss these results and implications for tool design and code review.

show abstract

Biases and differences in code review using medical imaging and eye-tracking: genders, humans, and machines

Cited by 22 publications

References 107 publications

Expert Programmers Have Fine-Tuned Cortical Representations of Source Code

Expert Programmers Have Fine-Tuned Cortical Representations of Source Code

Women's Participation in Open Source Software: A Survey of the Literature

Artefacts Package - First come first served: the impact of file position on code review

Contact Info

Product

Resources

About