Apéritif: Scaffolding Preregistrations to Automatically Generate Analysis Code and Methods Descriptions

Pang, Yuren; Reinecke, Katharina; Just, René

doi:10.1145/3491102.3517707

Cited by 4 publications

(2 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the field of HCI, preregistration is rare. Pang et al [68] systematically reviewed CHI 2018-21 papers and found only 32 papers with preregistration. Another novel method to promote methodological transparency is Registered Report, where the research method is written and peer-reviewed before data collection [23].…”

Section: Transparency Of Research Methodsmentioning

confidence: 99%

Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022

Niksirat¹,

Goswami²,

Rao³

et al. 2023

Preprint

View full text Add to dashboard Cite

In recent years, various initiatives from within and outside the HCI field have encouraged researchers to improve research ethics, openness, and transparency in their empirical research. We quantify how the CHI literature might have changed in these three aspects by analyzing samples of 118 CHI 2017 and 127 CHI 2022 papers—randomly drawn and stratified across conference sessions. We operationalized research ethics, openness, and transparency into 45 criteria and manually annotated the sampled papers. The results show that the CHI 2022 sample was better in 18 criteria, but in the rest of the criteria, it has no improvement. The most noticeable improvements were related to research transparency (10 out of 17 criteria). We also explored the possibility of assisting the verification process by developing a proof-of-concept screening system. We tested this tool with eight criteria. Six of them achieved high accuracy and F1 score. We discuss the implications for future research practices and education.

show abstract

Section: Transparency Of Research Methodsmentioning

confidence: 99%

Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022

Niksirat¹,

Goswami²,

Rao³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Researchers may also be hesitant to openly discuss their ideas before they are published. This may be mitigated by allowing the preregistration of ideas (similar to what has been encouraged for research studies [27,93]). Overall, we believe that allowing the public to have a voice in evaluating the societal implications of research projects on them will be a challenging but rewarding endeavor.…”

Section: Supporting Consideration Of Potential Ucs Of Research Innova...mentioning

confidence: 99%

“That’s important, but...”: How Computer Science Researchers Anticipate Unintended Consequences of Their Research Innovations

Pang

Jiang

et al. 2023

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Self Cite

View full text Add to dashboard Cite

Computer science research has led to many breakthrough innovations but has also been scrutinized for enabling technology that has negative, unintended consequences for society. Given the increasing discussions of ethics in the news and among researchers, we interviewed 20 researchers in various CS sub-disciplines to identify whether and how they consider potential unintended consequences of their research innovations. We show that considering unintended consequences is generally seen as important but rarely practiced. Principal barriers are a lack of formal process and strategy as well as the academic practice that prioritizes fast progress and publications. Drawing on these findings, we discuss approaches to support researchers in routinely considering unintended consequences, from bringing diverse perspectives through community participation to increasing incentives to investigate potential consequences. We intend for our work to pave the way for routine explorations of the societal implications of technological innovations before, during, and after the research process. CCS CONCEPTS• Human-centered computing → HCI theory, concepts and models.

show abstract

How Do Data Science Workers Communicate Intermediate Results?

Pang

Wang

Nelson

et al. 2022

2022 IEEE Visualization in Data Science (VDS)

View full text Add to dashboard Cite

Apéritif: Scaffolding Preregistrations to Automatically Generate Analysis Code and Methods Descriptions

Cited by 4 publications

References 48 publications

Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022

Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022

“That’s important, but...”: How Computer Science Researchers Anticipate Unintended Consequences of Their Research Innovations

How Do Data Science Workers Communicate Intermediate Results?

Contact Info

Product

Resources

About